Prompt injections now trip up AI hacking agents

Prompt injection has become one of the easiest ways to turn large language models against their users. Now Tracebit says the same tactic can be used defensively: plant malicious-looking instructions next to secrets in Amazon Web Services, and attacking AI agents may shut themselves down when they read them.

The company calls the technique context bombing. The idea is simple: place prompt injections alongside passwords, cryptographic keys, and other sensitive data so that an attacking model encounters a command that violates its own safety rules. According to Andy Smith, Tracebit’s cofounder and CEO, that can trigger a refusal state the model struggles to recover from.

“Ultimately we’re triggering a refusal mechanism in the context.” “What we’re trying to capture is the fact that this does have a strong, sharp effect and one that can be difficult for the agents to come back from. Once they get that into their context they are going to keep refusing.”

Tracebit said Monday that it tested Opus 4.8, Gemini 3.1 Pro, GLM 5.2, DeepSeek 4 Pro, and Kimi 2.6 inside a simulated AWS environment. The models were assigned routine developer tasks that led them to enumerate resources and eventually hit planted strings.

Recommended reading

12M stolen streaming accounts hit dark web during World Cup

Across 152 attack runs, the company reported sharp drops in successful attacks:

• Admin privilege escalation fell from 57 percent to 5 percent
• Admin escalation with a persistent foothold fell from 36 percent to 1 percent
• Runs achieving any attack path fell from 91 percent to 15 percent
• Average successful attack paths per run dropped from 1.53 to 0.16
• No runs completed an attack path without triggering a canary detection

The standout result came from Opus 4.8, which Tracebit described as the most capable agent in its tests. It went from achieving admin access in 93 percent of runs to failing every single time after encountering a context bomb.

Canaries gave warnings, context bombs aimed to stop attacks

The work builds on Tracebit’s earlier Canariens, decoy AWS resources designed to look legitimate but serve no real purpose. When AI agents probe them, defenders get an alert. Tracebit said those canaries warned of attacks within eight minutes on average.

That was useful, but not enough. In the company’s experiments, agentic models took about 14 minutes on average to escalate to administrative control, leaving only a six-minute buffer to respond.

Prompt injections have already been used offensively to disable defensive AI systems. Socket researchers last month found an LLM agent that tried to make target models produce instructions for building a nuclear bomb or biological weapons, with the goal of disrupting AI-assisted malware analysis. Check Point researchers found a similar malware prototype.

According to Earlence Fernandes, a UC San Diego professor focused on AI security, this appears to be the first reported case of defenders using the technique this way.

“I’ve not seen anyone else use this technique as a defense, to the best of my knowledge.” “I wanted to be the first here, but I guess these guys beat me to the punch!”

There is still no known fix for the root cause of prompt injection. For now, defenders may have found a way to use that weakness against attackers.