Three Small Changes, Dramatic Outcomes

01Where Episode 02 left us

At 9:10 on Monday, the reconciliation team opens a spreadsheet comparing four models.

One column is green. The production failure list is unchanged.

The agent still repeats malformed calls, returns fields the downstream system cannot use, and chooses the wrong operation when tool names overlap. In the most serious failure, it processes 847 of 2,347 eligible invoices and stops with 1,500 still waiting.

Episode 01 taught the team to find the first broken contract. Episode 02 showed where that contract lives: Identity, Memory Policy, Orchestration, Interception, or Observability and Evals.

The team can now draw the machine. The next question is practical: what can it change this sprint without replacing the model?

02The sprint that changed no model

On Monday, the team freezes the model.

Validation errors will return to the next attempt. Outputs must follow an agreed shape. The model will see only the tools needed for the current step. The workflow will verify the ledger before it accepts completion.

By Friday, the lesson is not that four tricks worked. It is why they worked.

Each change removed a decision the model had been forced to improvise.

The title says three changes and the body teaches four patterns. That is deliberate. The four patterns are the controls the team ships. Section 10 reduces them to three deeper design moves: map the rules, preserve the handoff, and separate proposal from authority and proof.

Reliability improves when the system replaces a vague choice with a specific signal, boundary, or check.

03The whole pattern in one table

The model proposes. Reliability comes from the decisions before and after that proposal.

For business, this means defining the accepted outcome. For product, it means deciding where ambiguity is useful and where it is dangerous. Engineering implements the constraints. Security enforces authority. Operations keeps the trace and the recovery path.

04Pattern 1: retry with evidence

A naive retry says, “Try again.”

The model receives the same task, context, and uncertainty. A different answer may appear, but the second attempt has no better reason to succeed.

A useful retry explains what failed.

The useful version supplies four things: the field, observed value, expected rule, and correction scope.

A retry must change at least one condition:

• InformationReturn the exact validation error.
• StrategyRequire a different method after repeated failure.
• ContextRestart with a clean view and durable progress.
• AuthorityEscalate to another model, tool, or human.

If none changes, the retry is repetition.

Three failures are often blurred together:

One validator cannot do all three jobs.

The useful metric is not retry count. It is the share of failed first attempts that recover without producing a wrong business outcome.

05Pattern 2: constrain the output space

Teams often treat structured output as a developer convenience. Its product value is a smaller failure space.

Free text lets the model return anything it can express. A schema limits the response to fields and values the next system knows how to handle.

Consider the invoice output:

An instruction such as “return JSON” may still produce:

• Amount"$42.50"
• Due date"end of August"
• Status"awaiting payment"
• Extra fieldAn invented explanation the caller ignores

The response looks sensible to a person. The next system may fail or silently discard information.

A schema checks shape. An eval checks meaning. A completion rule checks the job.

OpenAI's Structured Outputs documentation distinguishes JSON mode, which produces valid JSON, from strict schema adherence, which constrains supported outputs to a supplied schema.^[1] Your application must still handle refusals, empty results, timeouts, unsupported schema features, and semantically wrong values.

A schema is a contract with every downstream consumer, not only the model. When it changes:

1. Version the new shape.
2. Identify affected consumers.
3. Test old and new cases.
4. Plan compatibility or migration.
5. Monitor empty, refused, and semantically invalid outputs.

A field added casually today becomes a silent mismatch three systems later.

Track schema validity and semantic correctness separately. A perfect shape can carry a wrong answer.

06Pattern 3: narrow the decision surface

Give an agent forty tools and it must distinguish forty operations before every action.

Some tools are irrelevant. Some overlap. Others use broad names such as query_data or update_record. The model must infer what each name means and whether the consequence is read, draft, or execute.

The goal is not always fewer total capabilities. It is fewer visible choices for this step.

A precise name is part of the control surface.

Three operations improve the catalogue:

1. Delete tools with no distinct job.
2. Rename vague tools to exact actions.
3. Reveal specialist tools only when the task needs them.

The third is progressive disclosure. The system keeps broad capability but presents a small, relevant set now. A reconciliation skill may expose invoice lookup, payment lookup, exception drafting, and completion checks. A supplier-onboarding skill exposes a different set.

Progressive disclosure creates a less obvious trade-off. Prompt caching rewards a stable request prefix, while changing the visible tool list can invalidate that prefix. CacheRouter, a preprint submitted on 24 August 2026, proposes separating tool discovery from the main model's fixed tool prefix. In its prototype, tested on 55 functional queries and one 30-turn dialogue, token-level cache hit rates reached 90.99% and 95.2%.^[7]

Those figures come from the authors' prototype, not an independent production benchmark. The transferable design question is narrower: can you reduce the visible decision surface without rebuilding the entire prompt on every turn?

A rare escalation tool may be essential. Review each tool on four dimensions:

The decision may be keep, rename, tighten, hide behind a skill, require approval, merge, or remove. The goal is not the shortest list. It is the clearest set that covers the workflow.

Near-neighbour tools fail at their boundary. Descriptions should therefore say when not to use them:

Use get_invoice_by_id when an exact invoice ID is known. Do not use it for vendor search or status lists.

Track correct-tool selection and steps per validated completion. Fewer calls help only when completion remains correct.

07Pattern 4: verify before exit

The invoice failure in Episode 01 did not involve malformed output or the wrong tool. The run stopped before the workflow ended.

Replace model confidence with an external checklist.

The model may propose completion. The harness checks the business state.

Output feedback says, “This field is invalid.” Completion feedback says, “This outcome is unfinished.” The first repairs a response. The second keeps the job open.

Anthropic's work on long-running agents uses feature inventories, progress files, startup scripts, and version history so a fresh session can reconstruct what remains.^[3]

The same pattern now appears in evaluation and product tooling. Braintrust's Harbor integration runs each task in an isolated container and records the verifier that inspects the container after the agent stops.^[6] Cursor's /goal, released on 19 August 2026, holds a long-lived objective until it is fully complete.^[8]

The products differ. The rule is the same: done is a state of the world, not a sentence.

A completion gate can create an infinite loop. Define:

• BudgetThe time and cost ceiling.
• AttemptsThe maximum attempts without progress.
• RecordThe durable progress artifact.
• EscalationWhere the run goes when it stalls.
• StopThe condition under which the task cannot complete.

“Continue until done” is not a policy unless both done and stop are defined.

08The measured receipt

LangChain held the model fixed and changed the system prompt, tools, and middleware, its term for hooks around model and tool calls.

On Terminal Bench 2.0, an 89-task coding benchmark, its reported score rose from 52.8% to 66.5%, a gain of 13.7 percentage points. The work included trace-based error analysis, self-verification, pre-completion checks, environment context, and loop detection.^[2]

The result does not prove that every harness change helps every workflow. It supports a narrower claim: changing the control system can materially change measured behaviour while the model stays fixed.

A second receipt concerns cost. In its 26 August 2026 newsletter, LangChain reported that Deep Agents v0.7 simplified the base harness and used 65% fewer base input tokens at comparable performance.^[5]

Both results are company-reported measurements on LangChain's systems. Neither predicts your workflow. Together, they identify the lever: with the model held constant, the harness changed both measured quality and input cost.

The reusable method matters more than either number:

1. Run a defined test set.
2. Read the failed traces.
3. Group failures by behaviour.
4. Change one system surface.
5. Keep the change only if evaluation improves.

09The feedback loop

A production trace records what happened. It becomes an eval only after the team states what should have happened.

A holdout set contains test cases the team does not use while designing the change. It checks whether the improvement extends beyond the examples that inspired it.

Twenty well-labelled cases from real failures may be worth more than a thousand synthetic cases no user has produced.

10Three changes, restated

Strip the four patterns back to their mechanism and three changes remain. Each removes a different kind of guesswork.

Change 1: give the agent a map, not a manual

A large instruction file feels like control. Over time, it becomes the opposite.

Old rules remain beside new ones. Everything looks equally important. The file consumes the context the agent needs for the job, and nobody can tell which instruction still earns its place.

Keep a short entry point that tells the agent where authoritative information lives. Store architecture, workflows, policies, examples, and tool guidance in smaller versioned sources. Load the detail the current job requires.

OpenAI reports moving in this direction after a central AGENTS.md became too large and stale. The transferable lesson is progressive disclosure: start with a map, then load the relevant source of truth.^[9]

A vague job invites the agent to invent work.

A preregistered coding-agent study by Sarel Weinberger and Amir Hozez, revised on 24 August 2026, tested an instruction many teams add by reflex: develop several approaches, compare them, and implement the best. Across six open-weight models and 4,644 runs, the instruction increased reasoning tokens by 2.4 to 7.4 times without improving success. Among discordant task pairs, the plain instruction won 26 to 13.^[4]

The result is specific to the study's models, tasks, and harness. It does not prove that planning is useless. It proves that extra reasoning steps must earn their place through end-to-end success, not through the appearance of diligence.

A useful job card names:

• ScopeWhat is included and excluded?
• AcceptanceWhat evidence proves completion?
• StopWhen must the agent stop, even if it could continue exploring?

Keep diagnosis and a final independent check. Do not invite open-ended exploration or repeated self-review unless evidence shows that they improve the complete job.

Change 2: make the handoff a checked artifact

A context window is temporary attention. It is not durable project state.

For work that outlives one call or session, store the plan, completed work, open questions, decisions, evidence, pending approvals, and next action outside the conversation.

A new run should begin by reading the handoff and checking its important claims against the environment. It should end by updating the artifact.

A weak handoff says:

Finished step three. Continue with step four.

A useful handoff says:

Updated the policy index to version 14. Retrieval passed for 28 of 30 cases. Two German-language cases still return version 13. Do not release. Evidence: retrieval-report-14. Next action: inspect locale routing.

The second handoff carries progress, uncertainty, evidence, and the next safe step.

Change 3: let the model propose, let policy authorise, let evidence close

Models are useful for proposing the next action. They should not be the final authority for actions that change important state.

Before execution, enforced policy should check:

1. which agent is acting,
2. who or what it represents,
3. the requested operation,
4. the resource being changed,
5. time and value limits,
6. whether approval exists.

The tool then runs inside network, file, credential, time, and spending limits.

After execution, inspect the real result. Read the record back. Run the test. Open the page. Check the transaction state.

Do not close the job because the model wrote “success.”

One-week experiment

11In practice: one sprint

Pick one high-volume model decision or one failure with serious consequences.

Do not ship all five moves at once if you cannot attribute the result. Start with the failure that appears most often or carries the largest consequence.

For the selected move, complete this record:

12Connecting the dots

The four patterns share one design principle: let the model reason over the uncertain part, but do not make it improvise the contract around the uncertain part.

A precise error reduces wasted retries. A schema protects downstream systems. A narrow tool surface reduces wrong actions. A completion check lets the organisation trust longer work.

Each constraint can also add latency, cost, maintenance, or blocked edge cases. A stable tool prefix may save cache cost while exposing too many choices. A completion gate may prevent premature exit while creating a loop. A strict schema may protect a downstream system while rejecting a legitimate new case.

That tension is the bridge to Episode 04. The next question is not whether constraints work. It is when they buy enough reliability, when they limit useful autonomy, and when a stronger model makes them unnecessary.

You now hold four ambiguity reducers: evidence in the retry, a versioned output shape, a narrowed tool surface, and an external completion check. You also hold the loop that turns one failed trace into a regression test. Episode 04 asks when each constraint buys reliability, and when it quietly costs autonomy.