AI test: 12 of 13 models knew new name, yet used old one

Imagine knowing the right answer but writing down the old one out of habit. That's what most AI models did in a recent benchmarking challenge. Twelve out of thirteen models, even though they knew the updated name of something, preferred to use the obsolete one. Think of it like a grandparent still calling "X" by its old name, "Twitter."

In the Kaggle Benchmarking Challenge, 12 out of 13 AI models demonstrated a preference for outdated names despite knowing the new ones. This test shows a certain "laziness" or resistance to change from machines. It seems even artificial intelligences have their quirks, much like our own old habits.

The main reason likely lies in how these models are trained. AIs learn from vast amounts of data, often collected over a specific period. If a name changes after their primary training, they might struggle to incorporate the new information, even if they're aware of it. It's as if they have an iron memory for the past, but find it harder to update the present.

The challenge highlights how AI models can struggle to update their knowledge bases, even with correct input. This isn't just about being particular. If an AI can't update a simple name, what happens with more complex, rapidly changing information? This trend raises questions about the reliability of their answers, especially when data freshness is crucial.

For those who use these technologies daily, the takeaway is clear: don't trust blindly. Even if an AI seems confident, it might be relying on information that isn't entirely fresh. A model might "know" the new name but still choose the old one, perhaps because it was more frequent in its training data.

This trend suggests users should always verify information provided by AI models, especially concerning rapidly evolving data. So, don't take everything an AI tells you as gospel. Use it as a smart assistant, but remember that the real brain is still yours. Or is it?