Harness Engineering · Episode 07

Organizations That Rebuilt Around Agents

Three layers have natural owners. The behavior between them still needs one.

Arc · The operating model Episode · 7 of 8 Next · Episode 08 — When the Harness Becomes the Habit
After this you will know
  • LocateWhy agent ownership falls between existing functions.
  • CompareThree internal operating models and the failure each creates when misapplied.
  • DefineThe decision rights a Harness PM or Agent Reliability PM needs.
  • OperateHow to turn human review into a service rather than a reassuring phrase.
  • KeepWhat judgment must stay in-house even when the infrastructure is rented.

01Where Episode 6 left us

Your Monday Morning Harness Kit gave the reconciliation team nine days of evidence gathering and four tickets.

The audit produced a system map, fifty reviewed sessions, a tool decision table, an eval-debt list, one maturity diagnosis, and a portability review.

It also produced a problem no ticket can solve alone.

Identity instructions sit with product. Memory and retry logic sit with engineering. Tool permissions sit with platform. Runtime boundaries sit with security. Ground truth sits with finance-domain experts. The eval threshold lives in a spreadsheet nobody officially owns.

Every component has a contributor. The resulting behavior has no single decision owner.

Episode 7 is about that gap.

02The org chart built for different software

Two quarters after the 2 AM invoice failure, the team has three agents in production and a fourth in testing.

The COO asks a new question:

If agents are now part of how work gets done, who owns their behavior?

The existing org chart has answers for three layers.

Applied AI or platform teams evaluate models. Platform engineering owns APIs and tool infrastructure. Security and infrastructure own identity, credentials, networks, and execution boundaries.

The decisions between them have no natural home:

Those decisions form the harness.

The first organizational change is not a new department. It is naming who has authority over those decisions.

An agent program has an ownership problem when several teams can change its behavior but no one can accept or reject that behavior as a product.

03The ownership map

Figure 01 · Concept
Three owners and a hole
ACCOUNTABILITY NEEDS AUTHORITY OVER BEHAVIOR MODEL Applied AI / platform Selection · routing · providers Cannot settle: is the workflow ok? TOOLS Platform engineering Contract · availability · build Cannot settle: allow this action now? ENVIRONMENT Security & infrastructure Identity · network · limits Cannot settle: did it meet intent? HARNESS · NAMED PRODUCT OWNER + ENGINEERING LEAD Decision rights, not a question mark Context policy Definition of complete Eval coverage Autonomy envelope Release readiness for behavior changes Coordination across model, tool, environment owners Several teams can change the behavior. One role accepts or rejects it as a product.
Read it as The three owned layers push their limits downward. What lands in the harness band is exactly what no existing function can settle alone.
Layer Natural owner Decision it can make Decision it cannot settle alone
Model Applied AI or platform Model selection, routing, provider integration Whether the full workflow is acceptable
Harness Named product owner plus engineering lead Context policy, completion, evals, autonomy, release Model, tool, and environment implementation alone
Tools Platform engineering API contract, availability, implementation Whether the agent should receive the action now
Environment Security and infrastructure Identity, network, storage, execution limits Whether the workflow outcome meets user intent

The harness owner is accountable for end-to-end behavior. The role does not absorb every adjacent function.

This distinction matters. Assigning everything to one person creates a fictional super-role. Assigning nothing creates shared responsibility without decision rights.

04Four decisions most organizations leave implicit

Who defines good?

A finance-domain expert may label the partial-payment case. The Harness PM owns the process that turns that judgment into an eval, keeps it versioned, and uses it at release time.

Who decides what the agent may do?

Security may define the policy. Platform may implement the permission. Product decides which action belongs in the workflow and which user promise it supports.

Who decides the work is complete?

Engineering implements the check. The workflow owner defines what business state counts as done.

Who can expand autonomy?

No single function should expand it unilaterally. Product proposes the value, security and operations assess consequence, engineering supplies evidence, and the accountable owner accepts or rejects the change.

If these decisions require a new committee each time, the organization has responsibility but no operating model.

05Three internal models

The useful variable is where workflow judgment sits.

Model 1 Platform-owned primitives

A central team owns shared infrastructure:

  • AccessModel providers and identity integration.
  • RuntimeDeployment and trace transport and storage.
  • ControlsCommon permission framework and tool standards.
  • MeasurementEval execution and cost accounting.

Product teams own domain rules, completion, escalation, and workflow evals.

How it failsThe platform team starts owning workflow decisions as well as infrastructure. Every product waits for the same backlog, local teams build shadow loops and tool wrappers to move faster, and the organization pays for the central platform plus the duplication it was meant to remove.

When it worksSeveral teams use genuinely common primitives, and the central group can operate them more reliably and cheaply than each team can. Operating signalProduct teams can change workflow context, completion, and evals without requesting a platform roadmap slot. If they cannot, the platform owns too much.
Model 2 Product-owned harness

Each product team owns its workflow behavior, orchestration, and evals. Shared infrastructure stays thin.

How it failsSix teams create six retry policies, six eval styles, and six failure taxonomies. A lesson found by one team does not travel, and provider outages, policy changes, or common tool defects require several separate repairs.

When it worksThe organization has a small number of different workflows. Customer support, contract review, code assistance, and incident response need different state, tools, latency, and approval rules, and local ownership keeps iteration close to the user outcome. Operating signalListen for repeated conversations beginning with “How are you handling…”. When several teams independently solve the same runtime or eval-infrastructure problem, a shared primitive is ready to be extracted.
Model 3 Harness as operating model

The agent and its control system are the product.

Product managers write behavior and autonomy specifications. Engineers build inside the control plane. Designers own interaction, refusal, progress, and review surfaces. Evals act as release criteria. Operations watches completed workflows rather than model uptime alone.

How it failsAn incumbent renames teams without changing code, incentives, release evidence, or decision rights. A transformation office appears. The operating model remains unchanged beneath new labels.

When it worksAI-native companies with one dominant product shape can design this way from the beginning. Operating signalIf eval review, autonomy review, and failure triage are not normal release rituals, the company has adopted the language, not the operating model.

06Choosing among them

Do not use fixed employee or team counts as universal thresholds. Choose using three questions:

QuestionFavors shared platformFavors product ownership
How similar are the workflows? Similar tools and controls Different data, outcomes, and risk
Where does iteration happen? Infrastructure changes dominate Domain behavior changes dominate
What repeats across teams? Runtime, identity, traces, eval execution Completion, escalation, domain judgment

Most mature organizations land on a layered version: platform-owned primitives with product-owned workflow harnesses. The interface between them matters more than the label.

07The Harness PM

Titles vary: Harness PM, Agent Reliability PM, Agent Product Lead, Applied AI Lead. Define the role by decision rights, not title.

Role definition
What the role owns — and what it does not
Owns
Workflow definition of good; eval coverage and the failure backlog; the autonomy envelope; harness roadmap and retirement decisions; release readiness for behavior changes; coordination across model, tool, and environment owners.
Does not own
Model training; every tool API; IAM implementation; runtime patching; incident command; domain judgment without the domain expert.
The RACI in plain language
DecisionAccountableRequired partners
Workflow quality and eval coverageHarness PMDomain expert, engineering, Applied AI
Tool authority and approvalsProduct and securityPlatform, legal, operations
Runtime, sandbox, credentialsPlatform, SRE, securityHarness team, vendor
Model routingApplied AI or platformProduct, finance
Behavior-change releaseHarness PM and engineering leadEvals owner, security when consequence changes
Incident commandSRE or securityHarness PM, business owner, vendor

The table prevents two bad outcomes: a PM accountable for controls they cannot change, or a technical platform with no owner for business behavior.

08What good ownership looks like

Avoid a vague “eval coverage percentage” unless the denominator is defined. Use observable signals:

  1. The top uncovered failure modes are known.
  2. Recent production failures became evals.
  3. Autonomy expansions cite evidence and rollback.
  4. Prompts, tools, policies, and completion rules have owners and versions.
  5. Temporary controls have retirement conditions.
  6. Product, platform, and security agree on who can stop or narrow the workflow.

A title without these decision rights adds another attendee to the meeting.

09Human in the harness

“Human in the loop” describes a synchronous checkpoint: the agent proposes, a person approves, the action happens.

That works at low volume. At higher volume, attention becomes a scarce resource.

Suppose 20,000 actions run each day. If 4% need review and each takes three minutes, the queue requires 40 person-hours a day.

Does the system route the cases where human judgment changes the outcome?

A human-review service needs:

MetricWhat it tells you
Escalation rateHow much traffic reaches people
Queue lengthWhether demand exceeds capacity
Review timeCost and user delay
Service levelHow long a case may wait
Override rateWhether routing is selecting useful cases
Reviewer agreementWhether the rubric is stable

Humans do not disappear. Their work moves toward ambiguity, consequence, and policy change.

This is human in the harness: people are designed into escalation, feedback, and governance rather than placed as a checkbox before every action.

10Skills need product ownership

A skill packages task-specific instructions, examples, and tool access that load when relevant.

Skills solve the progressive-disclosure problem from Episode 2. They also create a new lifecycle:

  1. Who writes the skill?
  2. Which eval proves it works?
  3. Who approves new tool access?
  4. How is it versioned?
  5. When is it retired?

Treat a skill as a product artifact, not a personal prompt file.

A small organization can assign ownership inside each product team. A larger one may use a rotating guild to define templates, review standards, and deprecation policy. The guild should not own domain correctness. It owns the quality of the skill-making process.

11What to keep, what to buy

Own what depends on your circumstances. Buy what improves mainly through vendor scale.
Keep in-houseBuy or reuse
Definition of goodModel APIs
Workflow and escalation rulesGeneric observability plumbing
Domain failure taxonomyRuntime primitives
Autonomy decisionsCommon tool libraries
Product-specific memory policyProtocol implementations
Release and retirement judgmentInfrastructure that does not differentiate the workflow

The boundary changes as vendors absorb common work. Review it instead of defending a permanent build-versus-buy ideology.

Episode 5 called the preferred posture “rent operations, own meaning.” This is the organizational version of that choice.

12Two companion boundaries

The eight-episode series focuses on the harness, the layer that decides.

Two adjacent boundaries deserve their own field guides.

The Tool Is the Contract

A tool is where model intent becomes an operation. Its contract includes more than input fields. It includes caller identity, authority, side effects, reversibility, idempotency, failure behavior, cost, and verification. Read the tool companion when your ownership problem centers on what the agent may request and how the result is proved.

The Environment Is the Product Boundary

The environment is where an approved action gains reach. Process, filesystem, network, identity, and persistence determine the blast radius. Read the environment companion — and the full Environment Engineering series — when the question is what execution can touch, what survives, and who owns recovery.

The bonus essays do not add two more harness clusters. They slow down at the two boundaries where decision becomes action and action becomes consequence.

13In practice: the ownership page

Take the one-page output from Episode 6 and add five sections.

Figure 02 · Practice
The ownership page
WHO DECIDES · WHO IMPLEMENTS · WHO VERIFIES · WHO APPROVES LAYER OWNER DECISION RIGHT EVIDENCE ESCALATION Model Applied AI Routing and provider Cost + quality run Product, finance Harness Harness PM + eng lead Completion, evals, autonomy Eval run + traces Business owner Tools Platform eng Contract and availability Verified result Security, legal Environment Security, SRE Identity and limits Containment log Incident command Shared primitives — platform Local judgment — workflow team Human review — owner, capacity, SLA TONIGHT TEST If a production failure happened tonight, can you name the four people? “Shared” is not an answer.
Read it as One page, four columns. Every row must resolve to a person or a role before the next autonomy expansion is approved.
SectionWhat to record
Layer ownersModel, Harness, Tools, Environment
Decision rightsWhat each owner may approve without committee
Shared primitivesWhat platform owns for all workflows
Local judgmentWhat the workflow team must control
Human-review serviceOwner, capacity, routing, service level
If a production failure happened tonight, could the team name who decides the fix, who implements it, who verifies it, and who approves wider rollout?

A “shared” answer is incomplete. Name people or roles.

14Connecting the dots

The org chart should follow the behavior the product must control.

Deterministic software concentrates product judgment before release. Agent behavior keeps changing with context, tools, models, and traffic. Product judgment therefore continues after release through trace review, eval updates, autonomy decisions, and retirement of old controls.

That is why the Harness PM sits between product management and reliability engineering. The role does not exist because prompts need a manager. It exists because stochastic behavior needs an accountable release authority.

Episode 8 opens that release loop. The finale returns to the invoice team one year later: the pager is quiet, half the original scaffolding has been deleted, and what the team preserved was not the first harness but the operating loop that decides what to test, what to change, and what to remove.

You now hold
Operating model 07 An ownership map across four layers, three internal models with their failure modes, a Harness PM defined by decision rights, and a human-review service with metrics.
The next question
What stays valuable when the model absorbs more of today's harness work?
Continue
Harness 08 When the Harness Becomes the Habit — the season finale on keeping the operating loop while retiring dead scaffolding.
Read alongside
Environment 00 The World Around the Agent — the boundary your security and infrastructure owners actually operate.
Sources
  1. LangChain — State of Agent Engineering.
    langchain.com/state-of-agent-engineering
  2. Anthropic — Response to the NIST RFI on agentic security.
    anthropic.com — NIST RFI on agentic security (PDF)