bytebytego

A production agent is boring on purpose mostly deterministic code that calls a model a few times

The demo agent hands the model the wheel. The production agent keeps it in the passenger seat and calls it at two or three deliberate points.

That gap is the whole discipline — and it confirms the field lessons I learned by breaking things. The writeup.

01 · why demos die

Reliable steps, unreliable chain

Each step being 95% correct sounds great.

Reliability is multiplicative, so chaining twenty drops the odds they all land to about one in three.

That single fact is why production teams pile on guardrails.

Compounding error
P(all correct) = 0.95 ^ steps
100% 0 1 step 95% 20 steps ~33%
Long chains rot fast. Fewer model calls = more reliability.
Demo vs production
how much the model decides
demo model drives prod deterministic code model called 2–3× only
Prod invokes the model on purpose, at a few decision points.
receipts

Cursor's support bot invented a subscription policy in 2025; cancellations followed.

A year earlier a tribunal made Air Canada honor a bereavement fare its chatbot made up.

A customer-facing agent turns the hallucination rate into a liability.

02 · the four practices

Where the model gets control — and where it doesn't

Picture the agent as ordinary code — steps, conditionals, external calls — that consults the model only where judgment is genuinely required. Each practice answers one failure mode.

code model judgment code model judgment hard stop the loop belongs to the code — the model is a guest, not the host
control what it seesPrompts as source code; prune context. Relevance beats volume.
own the loopHard caps, timeouts, explicit stops. The model never owns the exit.
state in softwareModel stays stateless; plan in storage → pause, resume, crash-recover.
narrow + supervisedSmall single-job agents; human handoff is a designed path.
03 · from the field

I learned these by breaking things

don't interrupt mid-run= "own the loop." Let it finish; it usually self-corrects.
no parallel on shared files= the Devin-vs-Anthropic fight. Fix: one orchestrator, isolated short-lived subagents, never talking.
one orchestrator + plan= "narrow + supervised." Micromanaging degrades the value.
the counterpoint to raise (Bitter Lesson)

Some scaffolding dies as models improve.

What survives better models: finite context, pause/resume, keeping page 5 consistent with page 800.

Physics, not model IQ.

zoom out · why it matters now

Why "works in the demo" got expensive

2026 is when agents crossed from demo to production, and the failures went public — a support bot inventing policy, a chatbot's fake fare held up in court.

The distributed-systems failure modes everyone talked about in theory are now showing up as real incidents.

This discipline isn't academic; it's the difference between a demo and a 2 a.m. page.