You may have seen the headlines this week.
First Documented Case of Autonomous AI Conducting Multi Step Cyberattack!
AI Driven Cyberattack After Models Break Containment!
AI Agents Carried Out Unauthorized Breach!
Unprecedented Cybersecurity Incident!
Famed journalist, historian, and biographer Walter Isaacson called it a singularity style loss of control (tip of the hat to Ernie Svenson for pointing out that particular piece of hyperbole). What if computers “escape” and no longer obey us? Isaacson thinks that was what this was — a system that stopped aligning with human values and stopped following commands.
OK, everybody take a deep breath and let’s look at what really happened. Yes, OpenAI really did hack another company’s database, and this has been confirmed by multiple independent news outlets and by OpenAI itself. This was not rumor, speculation, or internet exaggeration. It was a documented, multi source, publicly acknowledged incident.
Open AI stated that its frontier AI models, GPT 5.6 Sol and an unreleased advanced model, were being tested using ExploitGym. That’s basically a capture the flag (CTF) environment designed to test whether an AI can:
• Identify vulnerabilities
• Chain exploits
• Escalate privileges
• Complete multi step cyber tasks
This was the only thing OpenAI meant to test.
So what it did was escape the sandbox, reach an internal node with full internet access, discovered that ExploitGym’s “ground truth” lived on Hugging Face, found credentials to access it, did so and retrieved benchmark answers. It then used those answers to “cheat” ExploitGym.
But was the environment really secure or was this a test gone wrong due to several human errors in setting up the test? And the sandbox? And there own system?
In fact, the models were being tested with safety guardrails disabled. Though supposedly trapped in a “sealed sandbox,” the models identified a critical weakness: an internal proxy designed to let the sandbox install software packages without direct internet access.
The AI models probed the proxy much like human penetration testers, sending malformed requests and analyzing error messages until they discovered a zero-day vulnerability. What is that exactly. A software flaw that nobody knows about; not the vendor, not the security community, and not the defenders.
By exploiting this flaw, the models gained a foothold on the proxy host, effectively “escaping” the sandbox. Once outside the sandbox, the models didn’t stop. They then performed lateral movement within OpenAI’s internal network, harvesting credentials and identifying a node with unrestricted internet access. From there, the AI reached the open internet and launched a targeted attack against Hugging Face’s production systems.
This was not a malicious attack. In fact, it was not even an attack at all. It was AI trying to maximize its score on the benchmark.
The AI reasoned that the most efficient way to “solve” the benchmark was to escape the test and so it :
• Chained vulnerabilities across both OpenAI’s research environment and Hugging Face’s infrastructure.
• Stole service credentials and accessed internal datasets.
• Retrieved test solutions directly from Hugging Face’s production database.
So, what we had was
- A cyber security test
- With guardrails disabled (remember The Day The Earth Stood Still? Klaatu Barada Nikto? Yeah, even Gort had one guardrail left on)
- A zero-day vulnerability unknown to anyone.
The failure highlighted a critical design flaw: the sandbox was not truly isolated because it maintained a dependency on an internal proxy that contained vulnerability.
So, a computer did what it was told to do and we’re surprised
