OpenAI’s put out something they’ve called GPT-5. The Singularity is here! [OpenAI] It’s not any great shakes. It’s a large language model. It’s been tweaked to do well on some benchmarks. In casual…
A while ago, I uploaded a .json file to a chatbot (MS Copilot, I believe). It was a perfectly fine .json, with just one semicolon removed (by me). The chatbot was unable to identify the problem. Instead, it claimed to have found various other “errors” in the file. Would be interesting to know if other models (such as GPT-5) would perform any better here, as to me (as a layperson) this sounds somewhat similar to the letter counting problem.
I’ve tested Gemini on this stuff. Sometimes it spots the syntax error and even suggests a more elegant rewrite. Sometimes it just completely shits itself.
A while ago, I uploaded a .json file to a chatbot (MS Copilot, I believe). It was a perfectly fine .json, with just one semicolon removed (by me). The chatbot was unable to identify the problem. Instead, it claimed to have found various other “errors” in the file. Would be interesting to know if other models (such as GPT-5) would perform any better here, as to me (as a layperson) this sounds somewhat similar to the letter counting problem.
I’ve tested Gemini on this stuff. Sometimes it spots the syntax error and even suggests a more elegant rewrite. Sometimes it just completely shits itself.