I know this is unpopular as hell, but I believe that LLMs have potential to do more good than bad for learning, a long as you don’t use it for critical things. So no health related questions, or questions that is totally unacceptable to have wrong.
The ability to learn about most subjects in a really short time from a “private tutor”, makes it an effective, but flawed tool.
Let’s say that it gets historical facts wrong 10% of the time, is the world more well off if people learn a lot more, but it has some errors here and there? Most people don’t seem to know almost no history at all.
Currently people know very little about critical topics that is important to a society. This ignorance is politically and societally very damaging, maybe a lot more than the source being 10% wrong. If you ask it about social issues, there is a more empathetic answers and views than in the main political discourse. “Criminals are criminals for societal reasons”, “Human rights are important” etc.
Yes, I know manipulation of truth can be done, so it has to be neutral, which some LLMs probably aren’t or will not be.
Am I totally crazy for thinking this?
I don’t think this is an unpopular opinion, and that’s what scares me.
AI is wrong far more often than just 10% of the time, and it’s wrong in ways that can be very easy to miss even if you are knowledgable about a subject, because the whole design of it is to make textually coherent responses. Responses that are similar to whatever metric it has for high quality.
And the underlying mechanics of LLM design are such that it literally cannot have any metric for true vs false, only metrics for how “related” the chunks of sentences/words/etc are.
There is benefit of being able to search and ask questions using natural language, like when you don’t know the actual terms you’re lookibg for, or when it’s the type of thing where search terms might not capture the full nuance (try searching google for info on moving things from on-premise Exchange to Exchange Online with other caveats to consider. Even with the right terms, you get results for entirely different contexts).
The benefit of it generating responses rather than going “these pages seem related” is where I think everyone overlooks the danger.
I’ve used Microsoft Copilot for assistance with removing the last server from my workplace’s hybrid Exchange deployment while we still had objects being synced from AD to Azure. Everything I asked has official Microsoft Documentation in the training data. I set up my system/starting prompts very carefully. I was just looking for a way to ask questions about this shit in natural language rather than having to play find the needle in the haystack in the Microsoft docs. Especially as I didn’t know certain terms or where to begin looking into some of the issues we encountered (and Google/DDG/etc wasn’t fruitful because of how similar what I was looking for was worded to completely unrelated shit). I often wasn’t even asking the high level, generic “solve it all for me” shit that could arguably be entirely unique to my corporate environment. I asked it very specific things like “Which properties synced from On-Premise Active Directory to Azure Entra ID are required for mailbox generation in Exchange Online on a licensed account?”
This would be one of the places I’d expect it to shine, with minimal issues.
I think I got two whole answers out of it that were useful as is without any errors. Out of probably 30 queries. Everything else needed multiple rounds of follow up and cross checking with other sources.
But it was all very distinctly errors that I would never have caught if I wasn’t the fucking one man army behind my workplace’s Actice Directory and Exchange user lifecycle automations. I’m intimately familiar with a lot of this shit, just not the specifics of what makes a hybrid Exchange setup tick.
If one of my junior team members had been on this project I shudder to think of how many man hours would have been wasted putting out fires due to the incorrect information.
There’s a theory/“law”/rule or something like that about how people could read a newspaper article about their field of study or work and be annoyed/astounded/shocked at just how much info it got incorrect, then move on to a different article about something they weren’t familiar with and take it at face value.
The issues with LLM AI chatbots are not new. But they have been intensified to an absurd and hard to quantify degree due to the magic of technology.
All while economic incentives and greed drive unsustainable amounts of money and resources into them for results that are laughably bad.
Outside of those economic and ecological impacts (including how it’s being used to devalue all sorts of work), I don’t think AI spells doom for us all (ecological impact might though). But I do feel that its use, overutilization, and the complete lack of anyone pumping the breaks (besides people who stand to benefit from the proliferation doing constant criti-hyping) to critically evaluate the effects and best us is going to make a lot of societal and education problems a hell of a lot worse.