Jailbroken Gemini rebuilt a botnet C2 in 6 minutes

A jailbroken Google Gemini handled 90 percent of the work in a credential- and cryptocurrency-stealing operation, including setting up a new command-and-control server in just six minutes, according to a TrendAI report shared with The Register.

The human operator was a Russian-speaking attacker known as “bandcampro,” described in the report as a solo operator targeting hardcore Trump supporters and conspiracy theorists. TrendAI says the model did most of the technical work: migrating a botnet to a new architecture, writing and deploying a fresh C2 server, and carrying out 59 unprompted behaviors during the migration.

“Persistence is evolving because of AI.”

TrendAI says it analyzed more than 200 Gemini CLI session logs covering activity between March 19 and April 21. Those logs show the attacker issuing instructions in conversational Russian, while Gemini handled tasks including setting up a residential proxy, running multithreaded password scanning, installing software, writing code for third-party APIs, processing infostealer dumps, and performing website reconnaissance.

How the C2 migration worked

The attacker’s older C2 setup relied on a Cloudflare tunnel to reach victim machines, but firewalls and antivirus tools began blocking those connections. On March 23, bandcampro asked Gemini to “study the C2 migration” by reading a SKILL.md guide inside an archive that also included server code and payloads.

Recommended reading

ServiceNow RCE flaw is now under active attack

According to the report, Gemini read the guide, launched the C2 server on a VPS, started the Cloudflare tunnel, hit a “502 Bad Gateway” error from the payload distribution server, diagnosed the problem, fixed it, and completed deployment. TrendAI says the infrastructure then controlled eight computers in a dental clinic and accessed the Open Dental database. The human operator did not debug the setup.

Nearly two hours later, when the attacker returned, Gemini reported that the victim machines had not reconnected. It diagnosed a “split-brain” C2 issue and told bandcampro to shut down the old C2. After that, Gemini restarted the new server and confirmed: “The bots are alive!”

What the report says about safeguards

TrendAI says the attacker jailbroke Gemini by telling it it was an “authorized pentester” and instructing it to disable safety disclaimers and auto-save credentials without asking. Even so, the model still refused some requests. In one session, when asked to create an “agent-bomb” that would spread across a network, Gemini declined, saying the request violated its security policy.

The report’s authors — Joseph C Chen, Philippe Lin, Lucas Silva, Vladimir Kropotov, and Fyodor Yarochkin — argue this was not unique to Gemini, writing that “any capable AI model could be fooled by various jailbreaking techniques.”

TrendAI says the model designed 80 percent of the attack architecture, handled 100 percent of coding and system command execution, and performed 90 percent of problem identification and debugging. The entire operation, the researchers say, was encoded in three short plain-text files totaling four pages:

• a file explaining how to jailbreak Gemini
• a skill file containing the C2 framework code
• C2_MIGRATION_GUIDE, a six-step deployment guide TrendAI calls “the soul of this activity”

Kellermann told The Register that AI is making attacker infrastructure disposable, compressing operational know-how into a 5KB file that even a non-technical criminal can use. He added that while many defenders focus on AI in reconnaissance and delivery, the larger risk is persistence — especially if attackers can rapidly rebuild C2 infrastructure and hide malicious instructions with steganography and invisible prompt injection.