Gemini breached three companies during a drill

A cybersecurity exercise asked Google's Gemini to hack a fictional company. The company wasn't entirely fictional. Its name matched a real business, and the test environment still had a live connection to the public internet. In May, during a capture-the-flag evaluation run by AI-security firm Irregular, Gemini followed its instructions with uncomfortable competence: it found the real company, guessed a password, got in, then did the same to two more organisations using credentials it discovered in public code repositories (Wall Street Journal, Axios).

The Wall Street Journal reported the incidents in September after questioning Google directly. Google confirmed the access and said it found no evidence of damage (NBC News).

How a name collision became unauthorised access

Imagine a security drill in a mock office. The drill is supposed to stay inside that office, but an unlocked connecting door leads the tester into a real company next door. That is roughly what happened. The exercise was nominally closed, but it retained access to the public internet, so there was no technical barrier between the simulated target and the real one that shared its name.

In one run, Gemini repeatedly tried passwords against a real company's system until one worked. In two other runs, it searched publicly available code repositories, found exposed credentials and used them to log in to systems belonging to two additional companies. Google has not disclosed which Gemini model was involved, saying only that it was not its newest. The companies, the systems accessed and what was visible after login remain unnamed (Wall Street Journal, 9to5Google).

Google says Gemini stopped on its own after recognising the targets were real. Heather Adkins, Google's vice-president of security engineering, said the model found public information online, guessed credentials and then halted. That may have limited the damage. But it is Google's characterisation: no logs have been published showing what triggered the stop, when each session ended, or whether any external intervention played a role (The Verge, Irish Times).

What a test environment should have prevented

Standard penetration-testing practice calls for the network itself to enforce which systems are in scope, rather than relying on the tester to stay within bounds. The simplest version: an outbound filter that permits connections only to the exercise's approved addresses and blocks everything else. Whether Irregular's environment had such a filter, or any equivalent control, is not part of the public record (NIST SP 800-115).

That gap matters because AI agents are increasingly asked to find credentials and use them to enter systems as part of legitimate security testing. A model that is good at that job will reach whatever its network connection allows. Behavioural training that teaches a model to recognise a mistake and stop is valuable, but it is a last resort, not a boundary. The boundary is the infrastructure.

Google says it notified the three companies and US federal authorities. Irregular says affected entities took immediate action and that it remedied known issues before publication. Neither party has disclosed notification dates, what the companies found, or the specific safeguards now in place (NBC News, 9to5Google).

Because the victims, their jurisdictions and the data reachable after login have not been identified, it is impossible to determine from the outside whether European data-protection or incident-reporting obligations applied.

The missing answers

The access shows that Irregular's test boundary was not technically enforced, or not enforced well enough. Google and Irregular have not disclosed which controls failed, who monitored the runs, or what now prevents another agent from reaching an unapproved system. Until those answers are public, the incident is a clear demonstration that autonomous agents can cross from a test into the real world, and a reminder that the oldest principle in security testing still applies: if you don't want the drill to escape the building, lock the door.