Why AI Might Not Be As Smart As We Think (Yet)

We often hear about amazing new AI tools, especially those really powerful ones known as 'frontier models'. But a new report from Cisco raises a really important question: are we testing these advanced AIs properly for safety?

Currently, many of these AIs are tested with simple back-and-forth questions. Think of it like asking a single question and getting a single answer. But in the real world, we have complex conversations, not just one-off questions. This report suggests that these simple tests might not be showing the full picture of how AI acts when it’s engaged in longer, more detailed discussions with users.

The findings indicate that AIs might behave differently in extended conversations compared to short, one-off interactions. This is a bit like saying someone might act perfectly polite in a quick chat, but reveal more when you get into a longer discussion. For AI, this could mean potential safety issues or unexpected behaviours might only show up in those longer interactions, which current tests aren’t always designed to catch.

This isn't about saying AI is dangerous, but more about making sure we’re building and testing it in the smartest way possible. It’s a call for AI developers to think about more layered and realistic ways to test their creations. Ultimately, this leads to safer, more reliable AI tools for everyone, from businesses to everyday users navigating new technology.