Part 8 of Building My Agentic Workflow in Public. Part 7 audited a painful bill and found my own specs were the culprit. This is what round two of testing revealed: the plan was not wrong because it thought badly. It was wrong because it never looked.

After the retro in Part 7, I rebuilt my pipeline around a set of new rules and ran the whole thing again on the same two projects, SleemAI and Morpheus Observability. Round two went better, and I will share the full numbers in a coming post. But one finding from the audit was so simple, and so embarrassing, that it deserves its own short post.

The question the machine kept asking me

Quick context if you are new here: my pipeline is a chain of AI agents that build software in stages. A planning stage writes the specs, the documents that say what to build. Building agents implement them. Evaluator agents check the work. When a spec is ambiguous or two specs contradict each other, the run halts and waits for me to answer a question. Every halt costs tokens and time, because I have to come back, read the question, rule on it, and resume.

Round two still halted. So I did what Part 7 taught me to do, I refused to accept the halts as the cost of doing business and went to count them instead. I sorted every question the pipeline had stopped to ask me into buckets: which of these actually needed a human?

The answer: not even half.

Roughly 45 percent of the halt pain on one project traced back to questions that were already answered on the hard drive. Not answered in some deep, hidden way. Answered by one command. One find. One grep. Five seconds at planning time.

259 out of 486

Here is my favorite example, favorite in the way a bruise is a favorite.

One spec stated, confidently, that every plugin file in the codebase sits inside a directory named after its kind. It sounded reasonable. It was written in the same calm, authoritative tone as everything else in the document. Every downstream agent took it as fact and reasoned correctly from it. The building agents built for it. The evaluators evaluated against it.

Then a run collided with reality and halted, and I finally ran the one command that checked the claim.

Of 486 plugin files, 259 did not match the assumption. The majority. The spec was built on a guess, the guess was wrong more often than it was right, and the entire chain inherited it without anyone, agent or human, spending five seconds to look.

Thinks well, never looks

That is the pattern, and once I saw it I could not unsee it. The planning chain thinks well but never looks. The reasoning at every stage was genuinely good. Each agent drew correct conclusions from what it was told. The failure was upstream of all the thinking: the premise was never checked against the disk it was describing.

It is the consultant who writes a brilliant strategy for a warehouse they never visited. Humans do this constantly, plans built in meeting rooms on assumptions nobody walked downstairs to verify. My agents inherited the habit from me, because I was the one who let confident prose stand in for evidence.

flowchart TB
  A[Plan makes a claim<br>about the codebase] --> B{Did anyone look?}
  B -- no --> C[Every agent reasons<br>from a guess]
  C --> D[Run collides with reality,<br>halts, human pays]
  B -- "yes: one command,<br>one count" --> E[Claim carries a receipt]
  E --> F[Build proceeds]

The fix is one rule, not more process

My first instinct was to add a review stage. More checking, more agents, more paperwork. Part 7 already taught me where that road goes. The actual fix is one rule, and it is cheap:

Any claim a spec makes about existing code or data must carry the command that checked it and the number it returned.

Not “every plugin file sits in a kind-named directory.” Instead: “checked with find at planning time, 486 plugin files, 227 match the kind-named pattern, 259 do not.” The first version is a guess wearing a suit. The second version cannot quietly be a guess, because the receipt is attached. And when the receipt says 259 out of 486 break your assumption, the plan changes before a single token is spent building on it.

The rule works because it moves verification to the cheapest possible moment. At planning time, checking a claim costs one command. Three stages later, the same wrong claim costs a halted run, an expensive eval round, and a human context switch.

Trust, but grep

If you work with AI agents, or honestly with people, the takeaway travels well: a plan is only as good as its least-checked assumption, and the least-checked assumptions are usually the ones stated most confidently. Reasoning quality gets all the attention. Premise quality is where the money went.

So that is the rule I now hold every plan to, machine-written or not. Claims about the world come with receipts. Trust, but grep.

Round two produced real numbers on cost and quality, including a result on escaped defects I am still a little surprised by. That is the next post.