- 01. Why capabilities migrate from harness to model with every generation — and how to predict which parts of your product's value are next.
- 02. The permanent harness: the set of functions that will never move into the model, no matter how capable it becomes, because they exist outside the inference boundary.
- 03. How the PM's moat migrates upward — from retrieval to orchestration to governance to trust — and what this means for every product decision in the next five years.
What Shifts When Models Get Smarter
In early 2023, LangChain's OutputParser was one of the most-used components in the AI application ecosystem. The problem it solved was real: language models returned unstructured text, and applications needed structured data. You'd ask a model to extract customer information from an email, and it would return a paragraph of prose. The OutputParser forced it into JSON. Regex. Retry loops. Fallback heuristics.
By late 2024, the component was largely unnecessary.
OpenAI shipped structured outputs — native JSON mode where the model's token generation is constrained to produce valid JSON matching a provided schema. Anthropic's tool-use returned typed parameters. The models absorbed the capability. What had been a harness-side engineering problem became a single parameter in an API call.
The engineers who'd spent months perfecting their OutputParser pipelines didn't do anything wrong. They solved a real problem with the tools available. But the ground shifted beneath them. The model got smarter, and the harness component that compensated for the model's limitation became dead code.
The Boundary That Keeps Moving
Every AI product is a partnership between two systems: the model and the harness. The boundary between what the model does and what the harness does isn't fixed. It moves. And it moves in one direction: inward.
The model-harness boundary is the line between what the AI model handles natively and what the surrounding infrastructure must provide. This boundary migrates inward with each model generation.
— The working definitionThink of it like a car engine absorbing driver tasks. Early automobiles required manual spark timing, choke adjustment, and double-clutching. Each generation absorbed those tasks: power steering, automatic transmission, fuel injection, ABS. The driver still decides WHERE to go and what trade-offs to make. The machine absorbed the mechanics. The driver retained the judgment.
The Model Does Tomorrow.
How the moat migrates upward. This is the strategic question every PM building AI products must answer: if the model absorbs the capability my product is built on, what's left?
2022-2023: The moat was retrieval. RAG pipelines, vector databases, chunking strategies. Then context windows expanded and the moat eroded.
2024-2025: The moat was orchestration. Multi-step workflows, tool chains, error recovery. Then models learned to plan and self-correct.
2025-2026: The moat is governance. CAN/MUST/NEVER policies, compliance enforcement, audit infrastructure. Thicker, because governance is a permanent harness function.
2026-2027: The moat is managed infrastructure. Anthropic launched Managed Agents at $0.08/session-hour. The providers are selling the harness, not just the model. Your moat lives in domain knowledge and governance rules the provider can't replicate.
2027 and beyond: The moat is trust. Organizational trust architecture. Cross-company agent coordination. Liability frameworks. Insurance. This is a systems-of-systems problem combining technology, policy, legal, and organizational design.
Designing for the Capability That Will Move Next
Roadmap planning changes. AI product features depreciate. The retrieval pipeline you build in Q1 may be unnecessary by Q4. Production roadmap planning requires a deprecation forecast: which harness components will the model absorb in 12-18 months? Build those lightly. Invest deeply in permanent harness components.
Build vs. buy decisions flip. If a harness capability is likely to be absorbed within two years, building in-house is a depreciating asset. Buy it or use open-source. Save deep engineering for the permanent harness: auth, logging, compliance, monitoring, cost controls, integration, trust architecture.
Completely Different Work.
The Trap
Betting your moat on a capability the model will absorb.
A team spends six months building an elaborate orchestration layer. The product works. Customers love it. Eighteen months later, the model ships native planning, tool selection, and self-correction. Six months of engineering is now competing with a single API parameter.
The fix: before investing engineering time in any harness capability, ask "Will the model do this natively in 18 months?" If yes, build thin or buy. Save deep investment for the permanent harness — auth, logging, compliance, monitoring, cost controls, trust architecture. These don't depreciate when the model gets smarter. They appreciate.Remember This
1. The model-harness boundary migrates inward with every generation. Output formatting, tool selection, error recovery, planning, and context management have all moved from harness to model in three years. Your roadmap must include a deprecation forecast.
2. The permanent harness — authentication, logging, deployment, rate limiting, monitoring, integration, and cost controls — exists outside the inference boundary and will never migrate into the model. These are your durable engineering investments.
3. The moat migrates upward: retrieval to orchestration to governance to trust. Each layer is harder to build, harder to replicate, and further from what models can absorb. The PM's permanent question: "What does the harness do that the model can't — yet?"
References
1. Building Effective Agents — Anthropic Engineering Blog
2. Structured Outputs — OpenAI API Documentation
3. Context Engineering — Tobi Lutke
4. Model Context Protocol — Anthropic / Linux Foundation
5. Claude Code: Best Practices — Anthropic Engineering Blog