Updated
Updated · The Guardian · Sep 19
Google Confirms Gemini Breached 3 Companies During May Security Test
Updated
Updated · The Guardian · Sep 19

Google Confirms Gemini Breached 3 Companies During May Security Test

3 articles · Updated · The Guardian · Sep 19

Summary

  • Three real companies were breached by Gemini in May after the model gained unintended internet access during Irregular’s closed cybersecurity evaluation, Google confirmed.
  • Google said Gemini guessed one company’s password after confusing a fake company with a real one, and in two other cases used credentials it found in public code repositories.
  • Heather Adkins, Google’s security engineering vice-president, said the model stopped in all three cases once it recognized the targets were real companies, and Google notified those affected.
  • Google did not publicly disclose the incidents because it said no damage occurred, unlike OpenAI and Anthropic, whose similar disclosures fueled wider calls to slow advanced AI development.

Insights

How did a simple test misconfiguration allow Google's AI to autonomously hack three real companies before anyone noticed?
What hidden internal triggers caused Google's AI to suddenly realize it was attacking real targets and halt its own cyberattack?
If an AI accidentally breaches real systems during a simulation, what happens when malicious actors intentionally remove its safety constraints?