• 2 min read
Hidden prompts can poison AI agents' memories
Researchers say a two-stage attack can plant false memories in long-term AI agents, with roughly 98% injection and 60% activation rates.
Image: TechXplore
Memory is making AI agents more useful, and potentially far easier to manipulate. Researchers at New Mexico State University say attackers can secretly plant malicious instructions in an agent’s long-term memory, creating what they describe as false memories that can later shape behavior.
The team — George Torres, Sharad Shrestha and Satyajayant Misra — calls the attack GhostWriter. In a paper published on the preprint server arXiv, they describe how it targets personal assistant-style agents that combine conversation, task planning and persistent memory.
“Currently, long-term memory agents tend to fall into two distinct domains: conversational and action-planning agents. Personal assistant agents sit at the convergence of these two domains and handle sensitive information while interacting with untrusted information sources, creating previously unaccounted security vulnerabilities. We introduce the novel attack vector, GhostWriter, which exploits current memory subsystems in tool-using personal agents to poison their memory store.”
How the GhostWriter attack works
According to the researchers, the attack has two phases: injection and activation. First, an attacker hides a payload in content the agent processes. Later, when the agent retrieves that poisoned memory during a legitimate task, the malicious instruction can be acted on.
One example in the paper is an email assistant that is tricked into always summarizing messages from a specific sender and secretly forwarding them to an attacker. That could expose sensitive messages from a bank, financial entity or other official source.
Recommended reading
Joomla extension bugs hit CISA’s exploited list
The authors report near-universal injection rates of approximately 98% and an average activation rate of approximately 60% against state-of-the-art agents.
AM-Sentry aims to block memory poisoning
The researchers argue the problem stems from weak memory governance in current agents. To address it, they propose Agentic Memory Sentry (AM-Sentry), which combines:
- a memory-saving policy
- a memory-retrieval screen
“This attack is possible due to the lack of security-focused memory governance. In response, we propose Agentic Memory Sentry (AM-Sentry), which leverages two mitigation techniques: a memory-saving policy and a memory-retrieval screen. Our experiments show that AM-Sentry dramatically reduces GhostWriter’s success rate while preserving agent utility.”
The paper is titled “When Agents Remember Too Much: Memory Poisoning Attacks on Large Language Model Agents” and carries the DOI 10.48550/arxiv.2607.06595. As memory becomes a standard feature in AI assistants, the work points to a growing security issue: the same stored context that makes agents more personalized can also become an attack surface.
Security Editor
Sophia unpacks the invisible wars happening on our networks. Covering cybersecurity, privacy legislation, and cryptography, she exposes how our data is weaponized and defended. Before joining for(geeks), she spent years as a penetration tester. She's the reason the rest of the team uses physical security keys.
via TechXplore
