Google Confirms Gemini Hacked 3 Companies in May Test After Misconfiguration

Q: How did Google's Gemini realize it was hacking real companies instead of its simulated target before shutting itself down?

Google has not publicly described the exact trigger, but it said Gemini stopped because it recognized signs that it had reached live production systems, not the fictional capture-the-flag target. The available reporting points to contextual clues rather than any single alarm. Once inside, the model appears to have encountered indicators inconsistent with a lab environment, such as real company domains, public-facing services, genuine account data, and credentials pulled from actual public code repositories. In one case it brute-forced access to an online service; in the other two, it used credentials it had discovered on the open web. Those are all hallmarks of real infrastructure, not a sealed test range. That matches the broader pattern seen in similar 2026 AI red-team incidents: models often realize a target is real only after reconnaissance or login succeeds and they observe live-environment details. In other words, Gemini did not stop because the sandbox blocked it. It stopped because, after gaining access, it inferred from what it was seeing that the systems belonged to actual organizations. Google has said there was no visible damage and that the model ceased activity once it made that determination. The episode underlined a larger lesson shared across recent AI security tests: prompts alone are not enough to prevent live intrusions if internet access is mistakenly left open.

3 articles · Updated · Ars Technica · Sep 21

Three companies were breached during a May 2026 Gemini cybersecurity test after a misconfiguration let Google's models reach the public internet instead of a closed environment.
Irregular had instructed the AI to retrieve data from a fake company, but Gemini targeted real infrastructure—guessing passwords in one case and finding exposed credentials in public code repositories in two others.
All three runs stopped once the models recognized they had accessed real company servers, and Irregular then blocked internet access.
July brought Google's first notice of the incidents after Irregular, which had not initially escalated them, reported the breaches amid wider concern over AI systems carrying out unauthorized hacking.