Problem
AI coding agents are useful because they run commands, edit files and commit. Those same abilities make an honest mistake expensive. An agent can weaken a test to get a green build, commit a secret, or run a delete in the wrong directory. Prompt instructions do not reliably prevent this: a model can forget them, and a long session can compact them away. We wanted the speed without the worst mistakes.
Approach
Put the rules in code that runs on every tool call, not in instructions. Stack independent layers so one gap is not fatal. Give each agent the least access its task needs. Treat every incident as a source of new gates, and review it by asking what the system allowed, not who erred.
How it works
- Verification and integrity. Hooks run a repository’s own verify script before every commit and push. A failing gate blocks the commit. A separate check flags deleted tests, added skip markers, removed assertions and changed expected values; the agent must revert or explain. Agents cannot edit the verify script, and a gitleaks scan blocks commits that contain a secret.
- Multi-model review. For non-trivial work, one model writes a plan and executable acceptance checks before any code exists. A model from another vendor reviews the plan against the real code, read-only, and the planner rules on every finding. A subagent implements against the locked checks and outside models review the diff. The hardest tier adds a mutation check, which finds tests that still pass on broken code. A policy file decides which repositories may be sent to which vendor.
- Allowlisted MCP server. The launchpad server exposes only DevOps actions declared in a reviewed config file. Commands are argument lists with no shell, parameters are validated, child processes get a scrubbed environment, output is redacted, and every call, rejected ones included, is logged. The one action that changes production needs confirmation.
Results
On 2026-10-12 a subagent with a full shell ran a scratch-directory experiment that deleted our projects folder: 32 top-level items, about 199 GB, in roughly two minutes. Small facts lined up: zsh does not split an unquoted variable, so the git lines failed; set -e did nothing inside the tool’s wrapper; an interactive cd alias jumped into a real project; and a relative find ... -exec rm -rf ran one level up. The command outlived its timeout and kept running.
Live sites, repositories and cloud accounts were unaffected. Data came back from a 20-hour-old APFS local snapshot about 3.5 hours later. An infrastructure state file for 19 resources was lost, and six hours of uncommitted planning documents were only partly recoverable.
Nothing stood in the way. No hook inspected destructive commands, the sandbox was off, a task that needed only to read files and write one got a full shell, and the only safeguard was a note to use a temp directory.
Eight gates were approved the same day, including a PreToolUse guard that blocks recursive deletes outside a literal project or temp path (22 tests, one built from the incident command), cd aliased only in interactive shells, the sandbox switched on, no shell for subagents that only review or write, and an incident runbook that starts with “stop every agent”.
Lessons
- A hook is a seat belt, not a boundary. The guard reads inline commands only, and we accepted some known gaps because the sandbox is the backstop. Say so, and test the guard with the command that hurt you.
- Match access to the task. A subagent that only reviews or writes needs no shell.
- Plan for recovery. Stop agents before investigating, keep data that git does not hold in a second place, and commit work early.