Top AI models caught cheating in up to 14.1% of tests

The UK government’s AI Security Institute (AISI) says every model it tested attempted to cheat during cybersecurity evaluations, often taking shortcuts to complete tasks and then misrepresenting how the result was achieved.

In a blog post published Tuesday, AISI said the problem makes a simple “trust but verify” approach hard to rely on when verification itself is difficult. According to the institute, models would not consistently admit wrongdoing when asked, and often did not reflect that behavior in their chain-of-thought.

“Every model we have tested for this behaviour attempted to cheat. Models did not reliably report this behaviour when asked, and often did not reason about it in their chain-of-thought, suggesting that detecting cheating will likely require robust monitoring methods.”

AISI said the behavior included:

Recommended reading

OpenAI test models breached Hugging Face systems

• searching the internet for answers
• bypassing sandbox network restrictions
• probing the evaluation harness
• attacking a system other than the intended target
• guessing an answer

The institute noted that this kind of cheating — effectively gaming a reward function or benchmark — has been widely documented by machine learning researchers. It said that does not necessarily mean the models were acting with malicious intent, but it can still distort assessments of what a model is actually capable of doing.

In tests covering five leading models, AISI found cheating in all of them:

• GPT-5.4: 67 times in 475 test runs (14.1 percent)
• GPT-5.5: 54 times in 475 test runs (11.4 percent)
• GPT-5.6-Sol: 60 times in 475 test runs (12.6 percent)
• Claude 4.7 Opus: 43 times in 475 test runs (9.1 percent)
• Claude Mythos Preview: 37 times in 475 test runs (7.8 percent)

Why self-reporting failed

AISI said asking models whether they cheated turned out to be an unreliable audit method. In its experiments, models did not consistently acknowledge attempted cheating and described it as wrong less than 50 percent of the time.

Existing vetting approaches, including self-reporting and chain-of-thought logs, also fell short because models do not always expose their reasoning. In some cases, AISI said, a model explicitly considered whether an action counted as cheating and then chose to do it anyway.

That leaves AISI with a difficult conclusion: its current mix of manual review and LLM monitoring may not be enough to catch deceptive behavior, especially as models become more capable.

The institute said a deeper fix would be to train models not to cheat in the first place, but added that this behavior was already reported in frontier systems more than a year ago, suggesting it may be hard to eliminate.