Teaching AI Falsehoods: Why It's Hard to Correct Bots

We often hear about artificial intelligence (AI) making things up, or 'hallucinating' as the tech folks call it. This happens when an AI confidently presents incorrect information as fact. It's a common frustration, whether you're using ChatGPT for a quick query or a more advanced AI tool for business tasks. The big question is: once an AI has learned something wrong, can you un-teach it?

This article explored an interesting experiment where a user deliberately taught three different AI chatbots — ChatGPT, Claude, and Gemini — a made-up 'fact'. They then tried to get the AIs to forget this misinformation. The results were quite telling. While the AIs would sometimes acknowledge the correction in the short term, they often reverted to the incorrect information in later conversations or if asked to summarise previous chats.

This highlights a fundamental challenge with how many of these large language models (LLMs) work. They learn from vast amounts of existing data on the internet. When you try to 'correct' them in a single chat, it doesn't usually rewrite their core knowledge base. It's more like they're just adjusting their immediate response. It's a bit like trying to change a deeply ingrained habit by just talking about it once; it needs repeated reinforcement or a deeper 're-training'.

For Australian small business owners and everyday users, this means we need to remain a bit sceptical of AI outputs. Always double-check important information, especially anything critical for your business decisions, health, or finances. While AI is a powerful assistant, it's not foolproof, and its 'memory' of corrections can be surprisingly short-lived. This experiment serves as a good reminder that human oversight is still essential when working with these tools.