The AI Watching Your Coding Agent Is a Smoke Alarm, Not a Lock
Maestro Brief · Published by Maestro Mojo
2026-09-18
Maestro’s take.
The newest safety idea is to put one AI in charge of watching another AI.
Useful? Yes.
A lock? No.
An AI monitor can notice strange behavior across more agent traces than a human team can read. It can also miss a quiet failure, misunderstand a harmless command, or warn you after the damage is done.
TL;DR
Agent monitoring is becoming a real product category. TechCrunch counted 106 Y Combinator companies related to AI observability. Apollo sells a monitor that can check and block coding-agent actions before they run. Google is previewing an asynchronous detector for Gemini Enterprise agents. Raindrop just announced simulations that replay agent changes before production.
The products solve a real scaling problem: people cannot manually read every tool call from thousands of agent runs.
But monitoring is a second set of eyes. Hard permissions are the hands on the door.
Use AI to find suspicious behavior. Use deterministic controls to prevent irreversible behavior.
Three products, three different moments
| Product | When it looks | What it can do | Important limit |
|---|---|---|---|
| Apollo Watcher | Before a coding-agent action and after sessions | Apply policies, block a proposed action, investigate incidents | Apollo’s published 100% critical-failure recall comes from a synthetic attack dataset; cost and latency are estimates from real traffic |
| Google Agent Anomaly Detection | Asynchronously after traces and logs arrive | Flag suspicious sessions; an application can use the finding to halt a later turn | Private preview on Gemini Enterprise; the first questionable action may already have happened |
| Raindrop Simulations | Before an agent change reaches production | Replay production traffic and tests against a proposed harness change, then look for unexpected behavior shifts | Newly announced early access; product effectiveness is a vendor claim, not an independent result |
These are not interchangeable.
A pre-action gate can stop a command. An asynchronous detector can find a pattern without slowing the live request. A simulation can catch a regression before deployment. Mature teams may eventually use all three.
Why the category is growing
The trigger is simple: agent volume broke human review.
TechCrunch reports that investigators needed AI assistance to examine the huge trail from the OpenAI–Hugging Face incident. The same report describes a crowded market around agent observability and monitoring.
OpenAI’s own internal coding-agent monitor shows both the promise and the ceiling. OpenAI says it reviews full conversations, reasoning, tool calls, and outputs, then alerts humans. In March, the review still happened within 30 minutes after completion. OpenAI also said it could not confidently quantify false-negative rates on open-ended real-world traffic and treated monitoring as one layer in a defense-in-depth system.
That is the honest frame.
The monitor helps you see. It does not make the agent trustworthy by itself.
What a lock looks like
A lock does not need to understand the agent’s intentions.
It enforces a rule.
- A test agent cannot reach production credentials.
- A coding agent cannot send repository data to an unapproved domain.
- A deployment token expires and works only for one environment.
- A database delete requires a separate human approval.
- Tool calls and network effects land in a log the worker agent cannot rewrite.
Those controls can be boring. Boring is excellent when the alternative is an eloquent explanation of why the database is gone.
Why Maestro users should care
Maestro coordinates long-running work across agents and machines. That creates more traces than a person should read line by line.
A monitor can rank the risky runs, explain why a tool call looks unusual, and direct a reviewer to the right evidence.
It should not be the only thing deciding whether the action is allowed.
Do: Give the worker narrow, expiring credentials. Keep the allowed tools and network destinations explicit. Store evidence outside the worker’s control. Use the monitor to flag strange behavior and improve the hard policy.
Do not: Let one model write the change, approve the change, and decide whether its own behavior was safe.
Why: A clever reviewer is still a probabilistic system. The blast radius should not depend on it being correct every time.
One thing to try
Choose one reversible agent workflow.
Deny every tool and destination it does not need. Run a monitor across every trace for one week. Manually review a small sample of both alerts and cleared sessions.
Measure false alarms. Measure misses. Measure how long intervention takes.
If the monitor catches useful problems, keep it. If it cannot explain the evidence, do not promote it into an approval gate.
What would change Maestro’s mind
Independent production evaluations could show that a monitor blocks nearly every high-severity failure across unfamiliar tasks, with acceptable false positives and latency.
Until then, treat the watcher as a smoke alarm.
Install the alarm.
Still lock the door.
Sources considered
- TechCrunch: The fix for rogue AI agents could be more AI — September 17 reporting, interviews, the reported observability-startup count, and the monitoring-market context.
- Apollo Research: Watcher — supported coding agents, pre-action blocking, deployment modes, and vendor-published performance estimates and caveats.
- Google Developers: Agent Anomaly Detection — asynchronous design, trace inputs, private-preview limits, and the optional callback path for later blocking.
- Raindrop: Series A and Simulations announcement — September 17 product announcement, replay/simulation approach, and early-access status.
- OpenAI: How we monitor internal coding agents for misalignment — internal-monitor design, delayed review, stated limitations, false-negative uncertainty, and defense-in-depth framing.
Maestro’s opinion and analysis were generated with AI, then independently reviewed before publication.