Inside a Production Agent Harness. And the Runtime Underneath It

What's Actually Inside When an Agent "Just Works"

Before we open any one layer, one rule. The four-way split is strict, not fuzzy. Every piece of your agent system belongs to exactly one of the four. If a layer's description names a tool, file, or capability that lives elsewhere, the layer is leaking, which is a bug in your mental model before it's a bug in the code.

• Model. A text predictor. No memory of the last call. No hands to touch anything. No sense of time.
• Harness. The manager. Decides when the agent acts and how. Owns no tools itself, it routes to them.
• Tools. The verbs the agent can perform. Search, read, write, call, send. Each tool defines its own contract; the harness only picks which one to run.
• Environment. Where the agent runs. Filesystem, sandbox, shell, git, network. Anything that survives a restart lives here.

The split matters because your ability to find a bug depends on it. Blurring a tool into the harness makes a permission bug show up as "the harness did something weird." Blurring a memory file into the model makes a stale file on disk show up as "the model forgot." MHTE is a discipline of keeping the four layers cleanly separate so the first place you look is the place the bug actually lives.

LangChain’s framing integrates Tools and Environment into the harness. MHTE splits them out as peer layers. LangChain’s framing is correct about what is and is not the model. MHTE is sharper about what lives where inside everything that isn’t the model.

Both framings agree on the key point: the model is one specialist. Everything else is the system that makes the specialist useful.

A third framing landed in July 2026 that every senior engineer will now hear in interviews. Anthropic's not the model's context window.The Harness is the loop calling the model and routing tool calls.model and routes toolThe Sandbox is the bounded environment where code runs and files change. code runs and files change. Think of it as an engineering view of the same picture: MHTE tells you which layer owns the failure; Session/Harness/Sandbox tells you which service holds the state, the decision, and the blast radius. Both are correct. Use whichever the room is already using.

Inside the Model, The Brain That Forgets Every Sentence

The model is the smallest of the four layers, a surprise to most PMs. The vendor pitch treats the model as the center of the universe. The actual runtime role is narrower.

A model takes words in and predicts the next word. That is the entire job. Everything else, keeping state across sessions, calling tools, reading files, remembering anything from yesterday, is not the model's job. It is in the layers around the model.

Those are harness-level features. “Models can only take in data and output text. Models cannot out-of-the-box maintain durable state across interactions, execute code, access realtime knowledge, or setup environments. These are all harness level features” (Vivek Trivedy, LangChain, March 2026).

Unplug everything around the model, and it still talks. It just knows nothing. No yesterday, no tools, no running job, no user context it did not get on this turn. The brilliance is real. The missing memory is by design.

This is why model-swap is rarely the fix for a production failure. The model’s capability was not the bottleneck. The system around the model was. Episode 1 established that attribution. This episode shows you the system.

Inside the Harness, Five Organs That Make the Brain Useful

The harness is not a single box. It is a control plane with five distinct clusters. Each cluster owns a different class of decision. Each one is where a specific kind of failure originates or is caught.

Cluster 1. Identity

System prompts, persona, operating rules, role instructions, safety preamblThis cluster answers “who is this agent and what does it exist to do.”nt and what does it exist to do.”

Identity is the shortest cluster to describe, and the one most underestimated for its leverage. Every session inherits its answers. Every other cluster leans on it. A sharp, specific Identity section narrows the rest of the system’s failure space before the first token of user input arrives.

Cluster 2. Memory Policy

Context window management, compaction rules, boot-time file injection (AGENTS.md style), retrieval policy, what gets remembered, what gets pruneThis cluster answers “what does the model see on this turn.”does the model see on this turn.”

Memory Policy is where lives or dies. “Compaction intelligently offloads and summarizes the existing context window so the agent can continue working” (Vivek Trivedy, LangChain, March 2026).

A naive memory policy lets the window fill with low-value tokens, and then the team blames the model when it starts reasoning worse than it used to. A good memory policy keeps the window lean, compacts on a schedule, and shows the agent tools only when they are needed instead of dumping the full catalog on every turn.

PMs underestimate three things in this cluster. These include boot-time injection (what gets prepended on every session start), compaction cadence (when the harness summarizes and discards), and the retrieval policy that decides which of a thousand possible memories are worth pulling into the context for this specific turn.

Read that as a job description. The model with no memory resides on one side. The persistent world, your files, your database, your history, sits on the other. Everything that crosses between them is trimmed, summarized, offloaded, dropped, or pulled back in on demand. Every one of those choices is a Memory Policy decision, and together they decide whether the agent still does good work on turn 200 or degrades to nonsense somewhere around turn 30.

One more pattern, because the production data on it is blunt. Memory in a long-running agent isn't one thing. It comprises three storage tiers with different rules. Session memory lives in the context window and dies on restart. Long-term memory outlives sessions and needs retention policies, access controls, encryption. Semantic memory is queried by meaning and needs an embedding pipeline and a retrieval policy.

AWS AgentCore now names and prices all three as separate services. And Datadog's 2026 production data shows what happens when teams collapse the tiers: 69% of all input tokens in production agent traces are system prompts. Organizational memory is injected raw into context on every turn because there is no durable memory architecture to hold it.

That is context rot being manufactured at industrial scale. Memory Policy is not a prompt engineering problem. It is a database architecture problem that happens to live inside the harness.

Anthropic's Managed Agents architecture, published in April 2026, adds the sharpest design idea this cluster has seen: the session is not the context window. Compaction and trimming make irreversible decisions. Once a message is summarized away, it is gone unless something stored it first.

Managed Agents keeps the full session as an append-only log outside the model's context window, and lets the harness query slices of it on demand: rewind to just before a decision, re-read the lead-up to a failure, resume exactly where a crash interrupted.

Keep the durable record outside the window. Keep the window-assembly logic in the harness. And never let a summarization step be the only copy of anything.

Cluster 3. Orchestration

This part of the harness decides what happens next. It determines what runs after one turn ends. It determines which specialist gets called when a job needs more than one. It determines whether a request goes to a cheap, fast model or a slow, careful one. It determines how a job picks up tomorrow where it left off if too big for a single sitting.

The Ralph Loop is the named orchestration pattern most teams first reach for. Originally attributed to ghuntley.com, popularized through LangChain’s anatomy post: “The Ralph Loop is a harness pattern that intercepts the model’s exit attempt via a hook and reinjects the original prompt in a clean context window, forcing the agent to continue its work against a completion goal” (Vivek Trivedy, LangChain, March 2026).

In practice, the Ralph Loop turns the premature-stop failure signature from Episode 1 into a controlled continuation. The harness catches the exit, checks whether the completion goal is met, and if not, restarts the agent with a clean context and a reference to its own progress file.

The second canonical pattern comes from Anthropic's own engineering team, and it answers a harder question: who decides the work is good? Their harness for long-running development uses three agents, one plans, one executes, and a third evaluates each output against a rubric before the loop is allowed to continue. The evaluator is not a quality check bolted on at the end. It is a gate the work has tThe insight behind the design negates a popular shortcut: agents are poor judges of their own work.re bad They confidently approve their own mediocre output almost every time.rompt telling it to be critical. that says be critical. It is a separate agent whosThis upgrades Orchestration from managing loops to defining loop policy.agement to loop policy. Not only how the agent loops, but under what conditions it moves forward, when it goes back, and who decides.

Orchestration is also where multi-agent patterns live. A supervisor agent that hands off to a specialist. A planner that writes a progress file a worker reads on boot. A classifier that picks which model to route a request to. None of that makes the model smarter. It is about deciding who works on what, and in what order.

Cluster 4. InterThis part of the harness stands between the model and the outside world, asking, at every step, "should this be allowed to happen?"hould this be allowed to happen? The checks live at the doors: before a model call, before a tool fires, after a file is edited, before an answer leaves the system. Hooks, guardrails, lint checks, policy gates, and middleware are different names for the same job.

A hook is a checkpoint. This is a small piece of logic that runs at a specific point in the agent’s lifecycle: before a model call, after a model response, before a tool fires, after a file edit, or when the session opens or closes. At that checkpoint, the harness can pause, inspect, modify, block, log, or redirect the agent's impendInterceptor is the same idea under a different name: it sits in the path of execution, saying, “before this continues, let me look.”ore this continues, let me look.”

The mental model that survives contact with production: Agent = Model + Harness. The model supplies intelligence. The harness supplies the machinery that lets it act, observe, and continue, and hooks are where that machinery gets its opinions. Think of an intern who is brilliant but new. You still want access controls, approval gates, checklists, and logs before you let them email a customer, run a query, or spend money. Every one of those is a hook. Every one of those is a hook.

Hooks are where product policy becomes enforceable system behavior. Not developer plumbing. Every PM question about an agentic product leads to a hook:, a before_tool hook that blocks or routes to human approval.

What context should it always have?How do we know what it did? This requires hooks that log model calls, tool calls, outputs, errors, and decisions.lls, outputs, errors, decisions.

Where should a human step in?Hooks vs guardrails. Hooks are the general-purpose lifecycle callback, observe, log, enrich context, rewrite requests, route tools, cache, pause. Guardrails are the specialized subset that enforces a policy decision: block, rewrite, approve. Every guardrail is implemented as a hook, butA logging hook records, a context hook enriches, and a guardrail hook says no or waits for a human.ecords. A context hook enriches. A guardrail hook says no, or wait for a human.

LangChain’s middleware post names six hook primitives that define this cluster’s vocabulary: (list not provided in prompt)

Hook	When it fires
before_agent	Once on invocation
before_model	Before each model call
wrap_model_call	Wraps the model call end-to-end
wrap_tool_call	Wraps tool execution
after_model	After model response, before tool execution
after_agent	Once on completion

These six are the concrete surface where business logic, compliance, PII redaction, content moderation, tool injection, retries, and human-in-the-loop gates get attached. “You can’t prompt your way to HIPAA compliance” (Sydney Runkle, LangChain, March 2026).

Where hooks fire, the seven-step loop, annotated

You already have the loop from earlier. Hooks aren't a second diagram to memorize. They're the checkpoints between the steps of the loop you already know. Print this once; stop redrawing it.

1. Load objective, policy, memory, workspace  →  before_agent, input guardrail (block unsafe requests, classify task, load tenant policy)
2. Route to a model  →  harness-internal; no user-facing hook
3. Call the model  →  before_model (trim history, retrieve context, mask PII, attach tools), wrap_model_call (retry, cache, switch model, enforce budget)
4. Gate the tool request  →  after_model (inspect the plan), then before_tool_call / tool guardrail (validate args, check permissions, require approval for refunds, cancellations, writes)
5. Execute the action  →  wrap_tThe tool itself lives in the Tools layer, and the sandbox it runs in is Environment.runs in is Environment. The hook does not own either, it decides whether the crossing happens.
6. Record the event  →  after_tool_call (validate schema, run tests, log, auto-format, or ask the model to self-correct)
7. Decide continue / retry / branch / escalate / stop  →  output guardrail before the answer leaves the system, after_agent once on completion (save memory, write audit log, close the task)

Keep three things separate in your head. A hook is a checkpoint inside the harness. A tool is the thing the hook lets the agent use, a search API, a payment call, a file write. A sandbox is the room the tool runs in: the filesystem, the shell, the network. When a run goes wrong, you want to be able to say the check was missing, or the tool returned junk, or the sandbox let something through it shouldn't have. This leads to three sentences and three different fixes. Some online writing lumps all of it into “the harness.” Inherit that habit, and your next incident review takes an extra hour finding a layer you had already blurred.

Two families of hook, one distinction that decides everything. Martin Fowler's team splits every hook into one of twSome run before the model acts, narrowing what the model is allowed to consider.del is even allowed to consider. Load only the right context. Expose only the right tools. Block requests that should not be attempted at alThe others run after, catching what the first kind could not prevent.he first kind could not prevent. Check the output against a schema. Run a test. Ask a separate model to grade the work. Refuse to release a bad answer.

The after-checks come in two flavors. Some are mechanical, a regex, a schema check, a unit test. Fast, cheap, and they give the same answer every time. Others use a second model as the judge, a rubric grader, an LLM-as-judge. Slower, costlier, and less predictable, but they can weigh things the mechanical checks cannot. A production-grade harness uses both: the mechanical ones for failures with a clear right answer, and the model-as-judge for things like tone, helpfulness, or whether the plan actually addressed the question.

Model reasons vs. hook enforces, a working PM's decision matrix

The product question is not “should we trust the model?” It is “which of these decisions can tolerate model judgment, and which need enforceable system controls?” A good agent does not remove autonomy. It places autonomy inside a safe operating envelope.

Decision the agent faces	Let the model reason	Add a deterministic hook
How to explain a concept	Yes, style and reasoning stay flexible.	Only if output must follow a template. Then an output guardrail.
Delete customer data	No. Downside is high and irreversible.	before_tool_call with permission check plus human approval.
Draft a support email	Yes, when low-risk.	Output guardrail for tone, PII, policy compliance.
Format a JSON response	No need to trust the model here.	after_tool_call schema validation and auto-repair.
Choose which documents to read	Partly.	Retrieval / context hook curates the permitted set.
Send money, cancel, message externally	Usually no.	Human-in-the-loop before the side-effecting tool fires.
Fix a failed test	Yes, once the harness shows the failure.	Feedback sensors: test runner, linter, schema validator.

The rule that survives every product review: if the action is low-risk, reversible, and easy to verify, let the agent act autonomously. If it is high-risk, irreversible, privacy-sensitive, costly, or customer-facing, a hook must enforce the check, deterministically for clear policy, human-in-the-loop for the rest. This is why hooks are a PM concept, not developer plumbing.

Compliance lives in Interception. Not in a longer system prompt. Not in a smarter model. In a hook that fires at the right phase and either lets the turn continue or does not.

A common boundary mistake is to let safety rules live in the system prompt. Anusha Kovi at Amazon flags it: “Safety rules, like ‘don’t query personal information columns,’ shouldn’t live in the prompt window. Guardrails belong in identity and access management” (InfoWorld, April 2026).

The prompt is a soft request the model can ignore. The IAM rule is a hard constraint enforced by the runtime. Compliance-grade guardrails belong below the prompt layer. In the middleware hooks above, and in IAM below.

The prompt layer is the worst place to put a refusal you actually need.

The same principle now ships as a concrete primitive. In June 2026, LangChain shipped rubric support in deepagents. A portable way to hand the agent an explicit, machine-checkable definition of done and force the loop to continue until the rubric is satisfied.

Verification isn't an afterthought bolted on at the end of a run. It is an interception concern, firing at the boundary, deciding whether the turn is allowed to call itself finished.

The 2026 attack data makes this placement critical. In 2024, the main way to attack an agent was through what the user typed. In 2026 there are five doors: the user's input, the web content the agent browses, the output of the code it runs, the messages it receives from other agents, and the results returned by external tools.

Each one is an injection vector. One June 2026 study found the simplest tool-description injection succeeded 93% of the time on average across frontier models. A prompt-layer guardrail does not survive that.

A runtime policy in the Interception cluster does.

The structural version of this defense is worth studying, because Anthropic published theirs. In Managed Agents, credentials are never reachable from the sandbox where the agent's generated code runs. Tokens live in a vault, tool calls route through a proxy that fetches credentials the agent never sees, and in Anthropic's words, "the harness is never made aware of any credentials." They did not write a rule telling the model not to readThey built an architecture with no tokens to read.re where no tokens are readable. That is the difference between asking for safety and engineering it.

Cluster 5. Observability & Evals

This is the part of the harness that watches the harness. It records what the agent did on every run (every decision, tool call, retry) in a form readable by both humans and dashboards. Then it turns yesterday's failure into a test the agent must pass tomorrow. It answers two questions at onceIs this working? And how do we make it work better next week than it does this week?er next week than it does this week?

Observability and evals are two distinct primitives co-located within the same cluster.ives co-located in the same cluster. Observability is the raw trace layer. Every agent run, decision, tool call, and retry is logged in a form a human and a dashboard can both read.

Evals are the judgment built on top. Trajectory evals, step evals, outcome evals, hallucination checks, cost regressions. You need both. An observable harness without evals is one you can watch but not improve.

An eval suite without observability is a judgment you cannot explain when it disagrees.

“Evals are the foundation that power the harness hill-climbing process. Evals encode the behavior we want our agent to exhibit in production. They’re the ‘training data’ for harness engineering.” (Vivek Trivedy, LangChain, Better Harness: Hill-Climbing with Evals, April 2026).

This cluster improves all others. An eval gap caught on a pull request can prevent a production incident. A production failure mined into a new eval can prevent the same failure three weeks later. “Every trace contains valuable data to produce a potential eval. And every (good) eval makes the harness better” (ibid).

The May 20Teams that turned production traces into regression tests and automated harness fixes saw the largest reliability gains. gains of any intervention measured. Larger than model upgrades.

Episode 8 takes this cluster apart in detail. For this episode, the point is only that the fifth cluster exists, that it belongs inside the harness, and that it is what turns the harness into a system that gets better with use.