I do not believe that LLMs are intelligent. That being said I have no fundamental understanding of how they work. I hear and often regurgitate things like “language prediction” but I want a more specific grasp of whats going on.

I’ve read great articles/posts about the environmental impact of LLMs, their dire economic situation, and their dumbing effects on people/companies/products. But the articles I’ve read that ask questions like “can AI think?” basically just go “well its just language and language isnt the same as thinking so no.” I haven’t been satisfied with this argument.

I guess I’m looking for something that dives deeper into that type of assertion that “LLMs are just language” with a critical lens. (I am not looking for a comprehensive lesson on technical side LLMs because I am not knowledgeable enough for that, some goldy locks zone would be great). If you guys have any resources you would recommend pls lmk thanks

  • magic_lobster_party@fedia.io
    3·
    3 days ago

    Here are a few I’ve liked.

    Transformers from Scratch: https://brandonrohrer.com/transformers.html

    Goes through the mathematics behind transformer models, starting from the basics. I think it’s a good learning resource to get up to speed with the inner workings.

    Welsh Labs YouTube channel is a gold mine: https://youtube.com/@welchlabsvideo

    He’s very good with the visualization of deep learning models, and goes far and beyond other resources.

    Mapping the Mind of a Large Language Model: https://www.anthropic.com/research/mapping-mind-language-model

    I think this paper is quite interesting. It gives a clue on how ”language” is modeled in these models. What they’ve found is a ”golden gate” neuron in the Claude model. Amplifying that neuron makes it want to bring in Golden Gate Bridge into any conversation.