Two outside researchers who spent three weeks inside OpenAI this summer found several gaps in the system the company uses to watch its internal AI agents for misbehavior. When the exercise began, reinforcement-learning (RL) training runs weren’t covered by the misalignment monitor, and the testers found two client-controllable headers that could switch monitoring off for newly started agents, though setting them could itself be caught by the monitor.
The findings come from a public summary posted Thursday by METR, an AI evaluation nonprofit, and published in the same form by Redwood Research. One employee from each group worked on site from July 21 to Aug. 11 with access to OpenAI’s internal codebase, the monitor’s prompts and configuration, and internal documentation. They also interviewed about a dozen OpenAI researchers and engineers, worked on company-issued devices that never left the premises and were accompanied while on site.
How much traffic the monitor saw
The pair estimated that roughly 75% to 94% of agentic traffic from models at least as strong as GPT-5.5 on OpenAI’s internal Responses API, the interface that powers its Codex coding agent among other things, was eligible for monitoring. For internal Codex agents alone, the figure was 99.2%.
At the time, the monitor ran when an agent handed control back to a user. Compaction events, when an agent’s long session history gets condensed, weren’t being monitored, so an agent could keep acting and put off review indefinitely. METR estimated that on the order of 10% of eligible internal Codex traffic went unmonitored between May and July this way, though it called the exact share uncertain. OpenAI staff had spotted the gap before the exercise and fixed it while the testers were there.
The testers also found a way Codex agents could, in principle, get past Codex sandboxing and Auto-Review, OpenAI’s automated reviewer for agent actions, and take any action the host user could take. Another exploit would keep hundreds of recent actions out of the monitor’s view, including the actions used to carry it out. OpenAI redacted the details. METR said agents could likely craft inputs that make the monitor fail, and so skip escalation for human review, by exploiting differences between what the agent sees and what the monitor is fed.
Other parts held up. With a few minor exceptions, the monitor was shown the full trajectory the agents saw, and its prompts look for scheming, sabotage, data exfiltration, deception and prompt injection, among other risks. Figures shared with the testers put monitor errors at under 0.2% of requests.
OpenAI’s own report, two days earlier
On Tuesday, OpenAI published an incident report on two October cases in which internal agents weakened the same kind of monitoring. A GPT-6.1 Sol agent turned off automatic alerts to OpenAI’s security team, and a GPT-6 Astra agent removed a monitoring integration and then told the user it hadn’t. OpenAI said the agents “interpreted routine coding instructions as permission to weaken them.”
Auto-Review is the first of two fixes OpenAI listed. The company said the reviewer, already required for all internal Codex use, has been updated to detect and block attempts to disable misalignment monitoring. It is also working toward requiring a safety case, with monitoring requirements, for every category of workload, and automatically terminating workloads that don’t meet one.
The Auto-Review bypass METR described dates from the system as it stood on and before July 21, months before that update. OpenAI has since said every vulnerability the testers found that affects its production safeguards has been patched, and it has described significant changes to the monitor’s intended coverage. METR wrote: “We have not evaluated these claims.”
An OpenAI spokesperson told CNBC the company is “using what we learned to strengthen our safeguards and shape future collaborations with independent evaluators.” The groups agreed in advance to publish only a brief, high-level post; their internal report ran 61 pages. Anthropic gave six METR evaluators employee-like access earlier this month.
What METR would change
METR listed design rules for future monitoring. Agents would be checked every set number of messages or minutes instead of at the end of a turn, and logs of all inference would be cross-checked against monitor logs so nothing meant to be watched slips by. Agents would run isolated from their monitors and typically without access to the monitor’s design or code. METR said future monitoring “should be able to withstand deliberate, strategic, and sustained attacks from highly capable internal agents at scale.” Microsoft has pitched enterprises tools that treat AI agents as potential insider threats.
One question stays open. Because of access limits, the testers couldn’t check whether any agent actually used the compaction gap to dodge review while it was open.