Why Your Agent Fails

01The 2 AM failure nobody can explain

At 2 AM on Tuesday, the reconciliation agent reports that it is finished.

It processed 847 of 2,347 invoices. The dashboard is green. No exception fired. No tool timed out. Fifteen hundred invoices are still waiting.

The team blames the model. That is reasonable. The model is the part that spoke last.

But the model never saw 2,347 invoices.

The surrounding system fetched one page, passed 847 records into the session, and accepted the model's final sentence as proof that the full job was complete. Nothing compared the processed count with the source ledger. Nothing asked whether another page existed.

The model completed the task it received. The system represented the wrong task and accepted weak evidence.

The incident review now has three questions:

1. What state did the system show the model?
2. What counted as complete?
3. What checked that claim against the real world?

Those questions lead to code, evidence, and an owner. "The model was weird" does not.

The reconciliation team is a composite drawn from production patterns I have seen. It stays with us for all eight episodes. The details are illustrative; the failure mechanism is the lesson.

The model sets the ceiling of what an agent can do. The system around it sets the floor of what it does reliably. Users experience the floor.

02What is a harness?

A model turns the information placed in front of it into a proposed answer or action.

The harness is the control system around that model. It decides what information the model receives, which actions are available, what happens after the response, when a human must step in, and what evidence proves the work is complete.

Think of a strong contractor arriving for one shift. The model is the contractor. The harness is the work order, access badge, checklist, supervisor, and handoff log.

A better contractor helps. It does not replace the work order or the lock on the finance system.

OpenAI uses the term for the execution logic and repository design around Codex. Anthropic separates a durable session, the loop that calls the model, and the execution environment where tools run. The product details differ. Both make the same architectural point: useful agent behavior comes from a model inside a larger system.

03The four-layer map

Use four verbs.

When a run fails, find the first verb that departed from the user's intended outcome.

• Model failureThe model received correct evidence and reasoned badly.
• Harness failureThe harness supplied stale policy or accepted weak completion evidence.
• Tool failureThe tool used the wrong field or returned stale state.
• Environment failureCredentials, storage, network, or execution limits interrupted the work.

Each answer points to a different owner. A model swap does not repair pagination. A longer prompt does not enforce authorization. A new tool schema does not restore state that was never saved.

04One symptom can have four causes

A user says, "The agent forgot what I told it."

That sentence describes the experience, not the cause.

Start with the phase where the problem became visible. Then move backward to the phase where it became inevitable.

This is the first habit of honest attribution.

05Why benchmarks do not settle production reliability

A benchmark asks whether a model can solve a defined task under controlled conditions.

A production workflow asks whether the whole system can keep producing acceptable outcomes while context changes, tools fail, sessions restart, permissions differ, and completion depends on external state.

A model can classify invoices accurately and still sit inside a workflow that loses pagination state. It can choose a tool correctly in isolation and still loop when that tool returns a vague error. It can write correct code in one session and still call a week-long refactor complete because no progress record survives the reset.

Long work introduces questions short tests rarely answer:

1. Does state survive a restart?
2. Does the system notice stale context?
3. Does a retry receive new information?
4. Can it distinguish progress from repeated motion?
5. Does "done" mean the model stopped, or that external evidence passed?
6. Can a fresh session reconstruct the work without guessing?

Call this context durability: the system's ability to sustain useful behavior across many turns, tool calls, interruptions, and sessions.

Model capability contributes. The surrounding system carries the continuity.

More context is not automatically better context

A larger context window gives the model more room. It does not decide what deserves that room.

Research on context rot shows that performance can decline as context grows, even before the window reaches its limit. Old tool output, duplicated rules, stale plans, and partial summaries compete with the one fact the current step needs.

The product question is not "how much can we fit?" It is "which information is current, authoritative, and useful now?"

Episode 2 places that decision inside Memory Policy.

06The production gap is now mainstream

LangChain's 2026 State of Agent Engineering surveyed more than 1,300 practitioners. It reported that 57.3% had agents in production and another 30.4% were building toward deployment. Quality remained the most cited barrier, including accuracy, relevance, consistency, tone, and policy adherence.

The signal is straightforward. Getting an agent online is no longer enough evidence. The harder test is whether behavior stays acceptable as the workflow expands.

07Six failure shapes

You do not need a catalogue of every possible bug. Start with the shapes that recur.

Premature stop

The agent processes part of the queue and exits cleanly.

The clean exit is the clue. The system treated a final model message as completion evidence.

For invoice reconciliation, "done" should mean the processed count matches the eligible source count, every exception has an owner, and reconciliation checks pass.

AskWhat evidence allowed completion?

Infinite loop

The agent repeats a failed action or makes cosmetic changes that do not alter the constraint.

A retry must change something: information, strategy, context, or authority. Otherwise it is another sample from the same bad setup.

AskWhat changed between attempts, and what ends the loop?

Silent drift

The output is polished but follows an old objective or policy.

A coding agent follows a stale architecture note. An invoice agent applies last quarter's exception rule. Surface quality hides source failure.

AskWhich source of truth governed this run, and when was it validated?

Shared-vocabulary failure

"Active user" can mean a logged-in user to product, a billable seat to finance, and a contact to growth.

The model cannot settle a definition the organization never made explicit. The fix may be a governed glossary attached to the workflow.

AskWhere is the definition, who owns it, and which version reached the run?

Unauthorized action

Drafting a refund and issuing it are different authorities. Reading a record and exporting it are different authorities.

A prompt can state a rule. A consequential limit needs an enforceable check at the action boundary. The World Around the Agent (Environment Engineering 00) explains how process, network, identity, storage, and persistence bound what the action can reach.

AskWhich control permitted the action outside the model's own judgment?

Worker divergence

Parallel agents can perform their local tasks correctly and still produce an inconsistent whole.

Each worker sees a version of the task. Those versions drift. Two workers edit the same artifact, or one plans against a requirement another changes.

AskWhich shared state coordinated them, and what happened when their results conflicted?

08When a model swap is the right move

Model limitations are real.

Upgrade when the model cannot reason through representative tasks even with correct context, appropriate tools, and a clean evaluation.

Repair the surrounding system when the agent loses state, stops early, repeats a failed call, uses stale policy, crosses an authority boundary, or cannot prove completion.

Upgrade the model for a capability gap. Repair the system for a control gap.

A stronger model can make a weak control system harder to diagnose. It may produce a more persuasive explanation for stale data or continue a failing strategy with more variety. Better prose is not better evidence.

09Where prompt engineering fits

• Prompt engineeringImproves instructions for a model call.
• Context engineeringDecides what information reaches the model now.
• Harness engineeringControls the loop that assembles context, calls the model, routes tools, verifies results, and decides what happens next.
• Environment engineeringDefines what those actions can reach and what survives.

The disciplines are nested. Skill at the inner layer does not prove the outer layers are covered.

10The ownership gap

Most companies can name owners for three layers.

Applied AI or platform teams evaluate models. Platform engineering owns APIs and tool infrastructure. Security and infrastructure own identity, networks, credentials, and execution boundaries.

The decisions between them often lack one product owner:

1. What context reaches the model?
2. What counts as complete?
3. Which failure becomes an eval?
4. When may autonomy expand?
5. Which behavior change is safe to release?

One owner does not implement every component. One owner remains accountable for the behavior produced by all four layers.

Episode 7 develops that operating model.

11In practice: diagnose the last three failures

Take the last three sessions that failed or needed manual repair.

Use these definitions

• ModelCorrect context arrived, but reasoning failed.
• HarnessContext, memory, orchestration, completion, or verification failed.
• ToolOperation, schema, arguments, or returned state failed.
• EnvironmentExecution, identity, credentials, network, filesystem, or persistence failed.

If all three say Model, open the traces again. You may have recorded who spoke last, not where the failure began.

This episode gave you four labels and six recognizable failure shapes. The next question is what sits inside each box and which decision belongs where. Episode 02 opens the control plane, separates the model's working context from durable session history, and follows one file through the full lifecycle of a run.