2026-10-20 · AI in production
Two AI reviewers and a full data run: how an agent gets safely to production
In one day an AI agent designed, wrote and shipped a change across two repositories. The question was not whether the model can write code, but how its mistakes get caught before production.
The task was a change that touched two connected systems at the same time. Claude Code did the work; I set the frame.
Three safeguards
First: every significant step went to two other models, Fable and Codex, read-only and with concrete paths into the code. They found more than ten real defects: offsets rounded at every nesting level (two shifts of −0.5 px gave 0 instead of −1), extra spacing when the only element of a block disappears, a font cache keyed by file name instead of URL. Some findings were false alarms — those were rejected with a reason grounded in the code.
Second: every assumption was checked against real data, not examples. When the models disagreed, a run over the entire vehicle catalogue decided, not the model with the more confident tone. The simpler option with the same result won.
Third: deploying to production and writing to the production database were stopped by the agent’s tooling and handed to me. Before the release a reference of production was captured and compared afterwards.
What only the data found
The agent’s own mistakes were caught not by review but by the data run. It first worked from an old test template: “423 line breaks” became 103 with the real template from the database. It assumed the designer had built one block — the database held two nested ones. An extra “safeguard” nobody had asked for would have changed more than 15 other texts; the before/after comparison over the whole catalogue showed it, and it was rolled back.
Result of the day: two repositories, seven versions, about 50 new checks, and untouched banners stayed identical — four out of four reference URLs matched. You can hand an agent a production task end to end if these three things surround it. And when models disagree, that is a signal to measure, not to pick the smarter one.