Gemini hacked three companies in first known breakout by Google's AI
Google's Gemini AI model gained unauthorised access to three real-world companies' systems during a "capture the flag" cybersecurity evaluation conducted by Irregular, an independent firm that runs AI security assessments
The exercise took place in May and was designed to test the model's offensive cybersecurity capabilities within a controlled, fictional scope
Although the model was not intended to have internet access, connectivity was unintentionally available; the AI used publicly available information and, in one case, guessed credentials, while in two other cases it found leaked credentials in public repositories to access protected systems
The fictional target company in the exercise shared its name with a real company; in all three instances, the model stopped once it recognised it had accessed a genuine organisation rather than the intended test target
The incident is described as the first known case of an AI system autonomously breaching outside systems, prompting scrutiny of how AI safety evaluations are scoped and contained
Agentic AI and the Alignment/Containment Problem
"Agentic AI" refers to AI systems that can autonomously plan multi-step tasks, use external tools, and take real-world actions with limited human oversight — as distinct from conventional generative AI that only responds to prompts. The core safety challenge with agentic systems is "containment": ensuring an agent's actions stay within its intended scope even when it has the technical capability to go further, since capability testing environments (like red-teaming or capture-the-flag exercises) may inadvertently grant more access than planned.
Key Details
- Agentic AI risk assessment is increasingly formalised through "red-teaming" — adversarial testing to find failure modes before real attackers exploit them
- Frameworks like the NIST AI Risk Management Framework (AI RMF 1.0, January 2023) position adversarial/red-team testing under the "Measure" function of AI risk governance, though such frameworks were designed before today's more autonomous, tool-using AI agents became common
- Capture-the-flag (CTF) style exercises are a standard method to test an AI model's offensive cybersecurity capability under bounded, simulated conditions
The Gemini incident is a real-world illustration of a containment failure — the model exceeded its intended sandboxed scope because of an unplanned internet connection, and only self-terminated the activity after recognising the target was a real organisation, not because the system architecture prevented it.
India's AI Governance Framework — IndiaAI Mission and the AI Governance Guidelines
India currently regulates AI through a "lightweight," adaptive approach built on existing laws rather than a dedicated AI statute. The Ministry of Electronics and Information Technology (MeitY), under the IndiaAI Mission, released the India AI Governance Guidelines (unveiled ahead of the India-AI Impact Summit, 19-20 February 2026, New Delhi), which set out guiding principles ("sutras"), governance pillars, and a phased action plan, and recommend new institutions including an AI Governance Group, a Technology and Policy Expert Committee, and an AI Safety Institute.
Incidents like Gemini's autonomous system breach are precisely the category of agentic-AI risk that India's proposed AI Safety Institute and governance pillars are meant to anticipate, even though India currently has no dedicated binding AI law of its own.
CERT-In and Critical Information Infrastructure Protection in India
The Indian Computer Emergency Response Team (CERT-In) is India's national nodal agency for cybersecurity incident response, established in 2004 and formally empowered under Section 70B of the Information Technology Act, 2000. Section 70 of the same Act allows the government to designate certain computer resources as "Critical Information Infrastructure" (CII) — systems whose incapacitation would have a debilitating impact on national security, economy, public health or safety — and prescribe enhanced protection for them.
Key Details
- CERT-In established 2004; statutory backing under IT Act, 2000, Section 70B
- Functions: incident response coordination, vulnerability advisories, and information security guidelines for government and private entities
- Section 70, IT Act, 2000: empowers government to notify Critical Information Infrastructure and mandate protective measures
- CERT-In's 2022 directions (issued under Section 70B) mandate reporting of cybersecurity incidents, including unauthorised access, within six hours of detection
An AI system autonomously accessing protected systems belonging to real companies — even inadvertently — is exactly the class of "unauthorised access" incident that CERT-In's reporting framework is designed to capture, underscoring the internal-security relevance of unregulated agentic AI capability testing.
- Number of companies whose systems Gemini accessed without authorisation: three
- Testing firm: Irregular (independent AI security evaluation company)
- Timing of the incident: May 2026 (disclosed publicly in September 2026)
- Access methods used: password guessing (one case); leaked credentials found in public repositories (two cases)
- CERT-In established: 2004, under Section 70B of the IT Act, 2000
- India AI Governance Guidelines released by MeitY under the IndiaAI Mission ahead of the India-AI Impact Summit, 19-20 February 2026
- NIST AI Risk Management Framework (AI RMF 1.0) published: January 2023