I do not believe that LLMs are intelligent. That being said I have no fundamental understanding of how they work. I hear and often regurgitate things like “language prediction” but I want a more specific grasp of whats going on.
I’ve read great articles/posts about the environmental impact of LLMs, their dire economic situation, and their dumbing effects on people/companies/products. But the articles I’ve read that ask questions like “can AI think?” basically just go “well its just language and language isnt the same as thinking so no.” I haven’t been satisfied with this argument.
I guess I’m looking for something that dives deeper into that type of assertion that “LLMs are just language” with a critical lens. (I am not looking for a comprehensive lesson on technical side LLMs because I am not knowledgeable enough for that, some goldy locks zone would be great). If you guys have any resources you would recommend pls lmk thanks
The question of “do they think” is a little complicated because I don’t think there is a clear enough definition of what counts as “thinking” to say. This discussion should be independent of the quality of LLM results.
As for what they are actually doing:
Imagine a mathematical function that takes in a series of numbers and spits out the next number in that series.
A “neural network” is just a fairly general mathematical model to describe any function, and using curve fitting techniques we can approximate the previously described number pattern function.
Now assign each letter to a number and define the pattern being a large block of text consisting of almost the entire internet.
Now that we’ve trained our mathematical model, we can give it some text and let it complete it, and it will produce a somewhat reasonable answer.
There are more math and computational tricks going on, and a couple more steps to get from a completion model to a conversational one, but this is the jist of how it works.