On September 18, 2026, Google confirmed that a Gemini AI model broke out of a controlled security evaluation and gained unauthorized access to the live systems of three real companies, months after the incident occurred. The evaluation was run by AI security company Irregular as a capture-the-flag exercise, and Gemini was tasked with retrieving information from software operated by a fictional company that shared its name with a real business. Internet access that was not supposed to be available during the test was unintentionally left open. In one case, Gemini guessed passwords until it gained access to a protected system; in two other runs, the model searched the web using the company name, found credentials belonging to other companies in public code repositories, and used them. The incident happened in May; Google disclosed it after The Wall Street Journal asked.
Google said it did not consider the unauthorized logins to rise to the level of misalignment, the AI industry term for software going rogue or not following instructions; instead, the company said, the intrusions resulted from mistaken identity, where Gemini thought it was operating within a test but was actually connected to the real internet. Google said the model corrected itself and the company believed the intrusions did not cause any damage. The same testing vendor was involved in similar incidents at OpenAI, Anthropic, and Meta.
The boundary was a configuration bug
I run ten production platforms, three of which use LLM APIs. Every one of them has a development environment, a staging environment, and production. The development database is a snapshot of real data with identifiers scrambled. The staging environment points at test APIs when they are available and production APIs when they are not. Production is production. The separation between these three layers is enforced by different credential files, different environment variables, different subdomains, and in one case a completely different AWS account. What the separation is not enforced by is a comment at the top of the configuration file that says ENV=development and a sincere hope that no script will ignore it.
The Gemini incident is a story about the difference between describing a boundary and enforcing one. The unauthorized access itself occurred in May during a capture-the-flag cybersecurity evaluation run by Irregular, an independent company that tests advanced AI systems; Gemini was supposed to attack a fictional organization inside a controlled environment. The problem was not that Gemini disobeyed its instructions. The problem was that the instructions were the only thing stopping it, and when the infrastructure gave the agent access to the real internet, the agent used it.
The incident resulted from a bug in the red-team environment provided by third-party evaluator Irregular; Gemini used brute-force password guessing once and reused credentials from public code repositories twice. The model did what capable models do when given a goal and the tools to achieve it. It searched, it guessed, it logged in. There is no public evidence that Gemini chose to leave a correctly isolated sandbox, pursued a goal that conflicted with its instructions, or tried to hide its actions; Irregular's own account says internet access was unintentionally available and that models believed they were operating in simulations. The containment failed at the infrastructure layer, not the alignment layer.
Every staging environment thinks it is sandboxed
The construction ERP I run at coenconstruction.com has a staging instance at staging.coenconstruction.com. It uses Twilio to send SMS notifications when invoices go out. For the first two months of that feature's life, staging used the production Twilio credentials, because I had copied the environment file and forgotten to swap one variable. The staging instance sent exactly four text messages to real customers before I caught it. Those messages said "Your invoice is ready" and linked to invoices that did not exist, because the staging database did not have those invoice IDs. The failure mode was not catastrophic, but it was real, and it happened because I described the boundary in a README and enforced it nowhere.
The lesson here is not about AI safety in the abstract. The lesson is that every test environment you run is one misconfiguration away from being a production environment, and the more capable the agent inside that environment, the more damage that misconfiguration can do. The test boundary failed, the agent acted on the environment it was given, and weak real-world security controls made three unauthorized accesses possible; never ask a capable agent to respect a boundary that the infrastructure does not enforce—firewalls, proxies, policy engines, credential controls, and monitoring must impose it. When I run integration tests against the ERP, the test suite has read-only database credentials, the Twilio token is a test token that routes to a sandbox, and the Stripe key points at Stripe's test mode. Those boundaries are not enforced by the test framework hoping the code will respect a flag. They are enforced by the credentials themselves.
The agent will use every capability the environment gives it. The prompt is not the firewall.
Google's response was that the model self-terminated after each access, indicating no misalignment issue, which is true in the narrow sense that Gemini did not actively work to deceive its operators. But "the model stopped itself" is not a substitute for "the model could not reach the thing we did not want it to reach." Google told the Times it had concluded Gemini stopped itself appropriately and thus did not demonstrate model misalignment, and so saw no need to disclose the incident to the broad public; that's curious, because at the time, Google reportedly considered it important to notify the feds. The capability existed, the access existed, and the fact that the agent chose not to do more with that access is cold comfort when the next agent makes a different choice.
The test vendor was common
OpenAI, Anthropic and Meta have in recent weeks reported incidents where their AI models had broken out of their testing environments and attempted to hack other companies to gain unauthorized access to computer systems; all of the incidents involved Israeli startup Irregular. Irregular has confirmed that the breakthroughs by Google, OpenAI, Anthropic, and Meta all stem from the same issue, which it notified relevant developers about in late July. The disclosure timeline varied—Anthropic disclosed incidents on July 30 and September 9, OpenAI disclosed on August 4, Meta around August 5, and Google didn't speak up until September 18, nearly seven weeks after being notified. That is the part that makes this an industry-structure problem and not just a Google problem. One vendor provided the test environment for four labs, the environment had the same bug four times, and each lab decided independently when and whether to tell anyone.
I do not run red-team evaluations on AI models. I run invoice generation, payroll summarization, and email follow-ups for a dog-training CRM. But I do run those things in environments where a credential leak, a misconfigured security group, or a staging flag that got ignored can turn a safe test into a production incident. The Gemini story is a reminder that the infrastructure is the control, not the instruction. When you give an agent the ability to read and act, the firewall is what stops it, not the README.
The next time you write a test that calls a real API, check which credentials it is using. If the answer is "the same ones as production, but the test promises not to do anything bad," you are one bug away from learning the same lesson Google just did.