The alarming part is that such custom response to queries can be crafted into LLMs, further obfuscating the accuracy of anything an LLM produces. It is exactly the kind of mass manipulation that authoritarians have wet dreams about. This case is obvious but they all won’t be and how would an average user ever know the difference? They won’t. They’ll integrate falsehoods into their worldview and whichever LLM a group favours will show this manipulation in the aggregate behavior of the userbase.
Imagine a Donald Trump having that level of sophistication when he built up his Maga cult. Disinformation is already a big problem and they want these LLMs in schools for the purpose of brainwashing children in subtle and not so subtle ways toward whatever beleifs benefit the LLMs owners, not the children or the future of the nation.
The alarming part is that such custom response to queries can be crafted into LLMs, further obfuscating the accuracy of anything an LLM produces. It is exactly the kind of mass manipulation that authoritarians have wet dreams about. This case is obvious but they all won’t be and how would an average user ever know the difference? They won’t. They’ll integrate falsehoods into their worldview and whichever LLM a group favours will show this manipulation in the aggregate behavior of the userbase.
Imagine a Donald Trump having that level of sophistication when he built up his Maga cult. Disinformation is already a big problem and they want these LLMs in schools for the purpose of brainwashing children in subtle and not so subtle ways toward whatever beleifs benefit the LLMs owners, not the children or the future of the nation.