- LocateWhy agent ownership falls between existing functions.
- CompareThree internal operating models and the failure each creates when misapplied.
- DefineThe decision rights a Harness PM or Agent Reliability PM needs.
- OperateHow to turn human review into a service rather than a reassuring phrase.
- KeepWhat judgment must stay in-house even when the infrastructure is rented.
01Where Episode 6 left us
Your Monday Morning Harness Kit gave the reconciliation team nine days of evidence gathering and four tickets.
The audit produced a system map, fifty reviewed sessions, a tool decision table, an eval-debt list, one maturity diagnosis, and a portability review.
It also produced a problem no ticket can solve alone.
Identity instructions sit with product. Memory and retry logic sit with engineering. Tool permissions sit with platform. Runtime boundaries sit with security. Ground truth sits with finance-domain experts. The eval threshold lives in a spreadsheet nobody officially owns.
Every component has a contributor. The resulting behavior has no single decision owner.
Episode 7 is about that gap.
02The org chart built for different software
Two quarters after the 2 AM invoice failure, the team has three agents in production and a fourth in testing.
The COO asks a new question:
If agents are now part of how work gets done, who owns their behavior?
The existing org chart has answers for three layers.
Applied AI or platform teams evaluate models. Platform engineering owns APIs and tool infrastructure. Security and infrastructure own identity, credentials, networks, and execution boundaries.
The decisions between them have no natural home:
- ContextWhat the agent receives.
- CompletionWhat counts as done.
- EvalsWhich failures become tests.
- ToolsWhich appear for each workflow.
- ReviewWhen a human must look.
- AutonomyWhen it may expand, and which temporary control can be removed.
Those decisions form the harness.
The first organizational change is not a new department. It is naming who has authority over those decisions.
An agent program has an ownership problem when several teams can change its behavior but no one can accept or reject that behavior as a product.
03The ownership map
| Layer | Natural owner | Decision it can make | Decision it cannot settle alone |
|---|---|---|---|
| Model | Applied AI or platform | Model selection, routing, provider integration | Whether the full workflow is acceptable |
| Harness | Named product owner plus engineering lead | Context policy, completion, evals, autonomy, release | Model, tool, and environment implementation alone |
| Tools | Platform engineering | API contract, availability, implementation | Whether the agent should receive the action now |
| Environment | Security and infrastructure | Identity, network, storage, execution limits | Whether the workflow outcome meets user intent |
The harness owner is accountable for end-to-end behavior. The role does not absorb every adjacent function.
This distinction matters. Assigning everything to one person creates a fictional super-role. Assigning nothing creates shared responsibility without decision rights.
04Four decisions most organizations leave implicit
A finance-domain expert may label the partial-payment case. The Harness PM owns the process that turns that judgment into an eval, keeps it versioned, and uses it at release time.
Security may define the policy. Platform may implement the permission. Product decides which action belongs in the workflow and which user promise it supports.
Engineering implements the check. The workflow owner defines what business state counts as done.
No single function should expand it unilaterally. Product proposes the value, security and operations assess consequence, engineering supplies evidence, and the accountable owner accepts or rejects the change.
If these decisions require a new committee each time, the organization has responsibility but no operating model.
05Three internal models
The useful variable is where workflow judgment sits.
A central team owns shared infrastructure:
- AccessModel providers and identity integration.
- RuntimeDeployment and trace transport and storage.
- ControlsCommon permission framework and tool standards.
- MeasurementEval execution and cost accounting.
Product teams own domain rules, completion, escalation, and workflow evals.
How it failsThe platform team starts owning workflow decisions as well as infrastructure. Every product waits for the same backlog, local teams build shadow loops and tool wrappers to move faster, and the organization pays for the central platform plus the duplication it was meant to remove.
Each product team owns its workflow behavior, orchestration, and evals. Shared infrastructure stays thin.
How it failsSix teams create six retry policies, six eval styles, and six failure taxonomies. A lesson found by one team does not travel, and provider outages, policy changes, or common tool defects require several separate repairs.
The agent and its control system are the product.
Product managers write behavior and autonomy specifications. Engineers build inside the control plane. Designers own interaction, refusal, progress, and review surfaces. Evals act as release criteria. Operations watches completed workflows rather than model uptime alone.
How it failsAn incumbent renames teams without changing code, incentives, release evidence, or decision rights. A transformation office appears. The operating model remains unchanged beneath new labels.
06Choosing among them
Do not use fixed employee or team counts as universal thresholds. Choose using three questions:
| Question | Favors shared platform | Favors product ownership |
|---|---|---|
| How similar are the workflows? | Similar tools and controls | Different data, outcomes, and risk |
| Where does iteration happen? | Infrastructure changes dominate | Domain behavior changes dominate |
| What repeats across teams? | Runtime, identity, traces, eval execution | Completion, escalation, domain judgment |
Most mature organizations land on a layered version: platform-owned primitives with product-owned workflow harnesses. The interface between them matters more than the label.
07The Harness PM
Titles vary: Harness PM, Agent Reliability PM, Agent Product Lead, Applied AI Lead. Define the role by decision rights, not title.
- Owns
- Workflow definition of good; eval coverage and the failure backlog; the autonomy envelope; harness roadmap and retirement decisions; release readiness for behavior changes; coordination across model, tool, and environment owners.
- Does not own
- Model training; every tool API; IAM implementation; runtime patching; incident command; domain judgment without the domain expert.
| Decision | Accountable | Required partners |
|---|---|---|
| Workflow quality and eval coverage | Harness PM | Domain expert, engineering, Applied AI |
| Tool authority and approvals | Product and security | Platform, legal, operations |
| Runtime, sandbox, credentials | Platform, SRE, security | Harness team, vendor |
| Model routing | Applied AI or platform | Product, finance |
| Behavior-change release | Harness PM and engineering lead | Evals owner, security when consequence changes |
| Incident command | SRE or security | Harness PM, business owner, vendor |
The table prevents two bad outcomes: a PM accountable for controls they cannot change, or a technical platform with no owner for business behavior.
08What good ownership looks like
Avoid a vague “eval coverage percentage” unless the denominator is defined. Use observable signals:
- The top uncovered failure modes are known.
- Recent production failures became evals.
- Autonomy expansions cite evidence and rollback.
- Prompts, tools, policies, and completion rules have owners and versions.
- Temporary controls have retirement conditions.
- Product, platform, and security agree on who can stop or narrow the workflow.
A title without these decision rights adds another attendee to the meeting.
09Human in the harness
“Human in the loop” describes a synchronous checkpoint: the agent proposes, a person approves, the action happens.
That works at low volume. At higher volume, attention becomes a scarce resource.
Suppose 20,000 actions run each day. If 4% need review and each takes three minutes, the queue requires 40 person-hours a day.
Does the system route the cases where human judgment changes the outcome?
A human-review service needs:
| Metric | What it tells you |
|---|---|
| Escalation rate | How much traffic reaches people |
| Queue length | Whether demand exceeds capacity |
| Review time | Cost and user delay |
| Service level | How long a case may wait |
| Override rate | Whether routing is selecting useful cases |
| Reviewer agreement | Whether the rubric is stable |
Humans do not disappear. Their work moves toward ambiguity, consequence, and policy change.
This is human in the harness: people are designed into escalation, feedback, and governance rather than placed as a checkbox before every action.
10Skills need product ownership
A skill packages task-specific instructions, examples, and tool access that load when relevant.
Skills solve the progressive-disclosure problem from Episode 2. They also create a new lifecycle:
- Who writes the skill?
- Which eval proves it works?
- Who approves new tool access?
- How is it versioned?
- When is it retired?
Treat a skill as a product artifact, not a personal prompt file.
A small organization can assign ownership inside each product team. A larger one may use a rotating guild to define templates, review standards, and deprecation policy. The guild should not own domain correctness. It owns the quality of the skill-making process.
11What to keep, what to buy
Own what depends on your circumstances. Buy what improves mainly through vendor scale.
| Keep in-house | Buy or reuse |
|---|---|
| Definition of good | Model APIs |
| Workflow and escalation rules | Generic observability plumbing |
| Domain failure taxonomy | Runtime primitives |
| Autonomy decisions | Common tool libraries |
| Product-specific memory policy | Protocol implementations |
| Release and retirement judgment | Infrastructure that does not differentiate the workflow |
The boundary changes as vendors absorb common work. Review it instead of defending a permanent build-versus-buy ideology.
Episode 5 called the preferred posture “rent operations, own meaning.” This is the organizational version of that choice.
12Two companion boundaries
The eight-episode series focuses on the harness, the layer that decides.
Two adjacent boundaries deserve their own field guides.
A tool is where model intent becomes an operation. Its contract includes more than input fields. It includes caller identity, authority, side effects, reversibility, idempotency, failure behavior, cost, and verification. Read the tool companion when your ownership problem centers on what the agent may request and how the result is proved.
The environment is where an approved action gains reach. Process, filesystem, network, identity, and persistence determine the blast radius. Read the environment companion — and the full Environment Engineering series — when the question is what execution can touch, what survives, and who owns recovery.
The bonus essays do not add two more harness clusters. They slow down at the two boundaries where decision becomes action and action becomes consequence.
13In practice: the ownership page
Take the one-page output from Episode 6 and add five sections.
| Section | What to record |
|---|---|
| Layer owners | Model, Harness, Tools, Environment |
| Decision rights | What each owner may approve without committee |
| Shared primitives | What platform owns for all workflows |
| Local judgment | What the workflow team must control |
| Human-review service | Owner, capacity, routing, service level |
If a production failure happened tonight, could the team name who decides the fix, who implements it, who verifies it, and who approves wider rollout?
A “shared” answer is incomplete. Name people or roles.
14Connecting the dots
The org chart should follow the behavior the product must control.
Deterministic software concentrates product judgment before release. Agent behavior keeps changing with context, tools, models, and traffic. Product judgment therefore continues after release through trace review, eval updates, autonomy decisions, and retirement of old controls.
That is why the Harness PM sits between product management and reliability engineering. The role does not exist because prompts need a manager. It exists because stochastic behavior needs an accountable release authority.
Episode 8 opens that release loop. The finale returns to the invoice team one year later: the pager is quiet, half the original scaffolding has been deleted, and what the team preserved was not the first harness but the operating loop that decides what to test, what to change, and what to remove.
-
LangChain — State of Agent Engineering.
langchain.com/state-of-agent-engineering -
Anthropic — Response to the NIST RFI on agentic security.
anthropic.com — NIST RFI on agentic security (PDF)