The Self-Improving Moat

Opening scene

January 2025. Two enterprise AI products ship inside Fortune 500 customer-service organisations on the same week. Same frontier model under the hood. Same headcount on the engineering team — eleven people each. Both teams are good. Both have a former Stripe PM, two senior MLEs, a full-time evaluator. Both products launch at 64% resolution rate on tier-one tickets. The vendors brief their boards. The boards approve the spend.

Twelve months later, December 2026.

Product A is at 91% resolution rate on tier-one and 73% on tier-two. The curve is still climbing. The team has shipped 187 changes to the agent harness in the last quarter alone — most of them automated, a handful human-reviewed. The model under the hood has not been swapped. The infrastructure looks roughly the same as it did at launch. What has changed is the loop running underneath it.

Product B is at 47% and falling. The team has rewritten the system prompt four times in the year. They have switched models twice — first to a newer Anthropic version, then to a cheaper open-source alternative when the unit economics broke. Each switch produced a brief lift, then decay. Their churn rate has crossed 30%. The board has begun asking about the AI strategy.

Same model. Same team quality. Same starting point.

The variable was not talent. It was not capital. It was not the model selection. It was a single architectural decision made on day fourteen of the build: whether to invest the next two engineering quarters in fine-tuning the model, or in the loop that improves the system around the model. Product A picked the loop. Product B picked the model. Twelve months later, the gap is structural and widening.

This post is about that decision.

Central frame

The investment is in the loop, not the model. The compounding is in the discipline, not the technology. The operational depth — how to architect the loop, what to instrument, how to govern the changes — sits in the compounding feedback loops post earlier in this level. This post is the prior question: why is the loop worth building in the first place, when there are ten other things on the roadmap that look more urgent?

The short answer is that nine of those ten things will be matched by your competitors within eighteen months. The loop will not be — because the loop is not a feature. It is a discipline that pays in compound interest, and compound interest is the one financial structure most product organisations are constitutionally bad at funding.

The 60-second answer

The self-improving moat has three properties, and a system that lacks any one of them is not a self-improving moat — it is a manual improvement loop with marketing varnish.

One — production traces feed the next iteration mechanically. Not through quarterly business reviews. Not through a “lessons learned” deck after the bad quarter. The trace data flows into the harness on a daily or weekly cadence, with automated diagnosis surfacing the patterns that humans then approve.

Two — every failure becomes a documented harness edit that ships within days. A failure caught on Monday is a harness change live by Friday. The improvement is versioned in Git, validated against a holdout set, and the before/after metrics are visible in the dashboard. Reverts are one command away.

Three — the improvement velocity is measurable. Self-Optimisation Rounds per week is the leading indicator. World-class teams hit greater than 3.2 per week — about thirteen edits a month. Most teams sit at zero, because they shipped the model six months ago and called it done.

If your top AI feature has shipped fewer than twelve harness changes in the last four weeks, you do not have a self-improving moat. You have a static product wrapped around a probabilistic engine, and the second derivative is negative.

Why most moats decay

The structural argument starts with a hard look at the moats most product strategy decks are built on. In 2018 these moats held. In 2026 most of them are leaking.

Tech moats commoditise faster than at any prior point in software history. DeepSeek V4 ships in March 2026 at roughly one-quarter the per-token cost of Claude Opus 4 on most general-reasoning benchmarks. Eighteen months ago that gap took two years to close. Today the gap closes in two quarters. A tech advantage built on “we have access to the best model” has a half-life shorter than your product roadmap.

Distribution moats erode as channels shift. The cohort of products that ranked first on Google in 2019 and built a SaaS business off that traffic has watched their funnel collapse as the search surface itself changes. AI surfaces — Perplexity, ChatGPT shopping, Google’s AI overviews, vertical agents — are now the entry point for a meaningful share of intent. The distribution moat is not gone, but the channel under it is moving, and rebuilding takes years.

Brand moats fade with each generation. A brand built on “trustworthy enterprise software” in 2010 does not automatically extend to “trustworthy AI agent” in 2026. Buyers re-evaluate. The brand premium that justified a 30% price gap on a static SaaS product evaporates the moment the same buyer is asked to trust an autonomous agent making decisions in their workflow. Trust does not transfer across category — it has to be re-earned.

Patent moats expire on a clock. The clock has not changed. The pace of innovation around the patent has. A twenty-year patent filed today will be relevant for six of those twenty years if the underlying technique becomes obsolete in the wave that comes next. The 2026 PM cannot plan around the patent the way the 2008 PM could.

Data moats decay if you do not refresh them. This is the one most product teams misjudge. They look at their ten years of customer data and conclude they are protected. But customer behaviour shifts. The 2018 conversion patterns do not predict the 2026 ones, because the buyers are using AI tools to evaluate vendors and the input distribution has changed. Data without a refresh mechanism is a depreciating asset, and the depreciation is steepest in exactly the categories where AI is being deployed.

The pattern is consistent. The moats most strategy decks lean on assume a stable environment. The 2026 environment is not stable. Every classic moat is decaying somewhere on the curve.

The exception — moats that appreciate

Three moats actually get stronger over time. The first two have been understood for decades.

Network effects appreciate when each new user makes the product more valuable for the existing users. A two-sided marketplace with liquidity is the canonical example. Network-effect moats are powerful, but they require external conditions: a market structure that supports two-sided dynamics, a category where users genuinely benefit from each other’s presence. Most enterprise AI products do not sit in such categories.

Trust accumulation appreciates when each interaction without failure raises the cost of switching. A regulated bank that has run an AI compliance agent for three years without a violation has a moat that a new entrant cannot replicate in a quarter, because the new entrant has not yet survived three years. Trust moats are real, but they are also slow and partially out of your control — one bad incident can collapse the moat in a week.

The third is the one a single team can architect deliberately, on its own, regardless of category structure or external trust dynamics.

Self-improving systems appreciate by construction. Every interaction sharpens the next interaction. The improvement is mechanical. The compounding is in the architecture, not in the market. This is the moat any disciplined team can build, in any category, starting on the next sprint. And it is the one most teams do not build, because the investment does not show in the next quarter’s earnings.

The Karpathy Loop pattern

Tobi Lutke pointed Andrej Karpathy’s Autoresearch agent at Shopify’s templating engine in February 2026. The agent ran for nine days. By the end of the run it had identified 47 separate optimisation candidates in the rendering path, generated automated patches for all of them, and shipped 93 commits through the standard review pipeline. The result was a 53% improvement in templating render time on the production workload. The original PM team had been working on rendering performance for eighteen months and had hit a plateau.

The Autoresearch repository hit 42,000 stars on GitHub within six weeks, and Aakash Gupta’s PM Guide framed the pattern as the Karpathy Loop — Fortune picked up the framing in their April issue. The pattern has since become shorthand for the broader move.

The move is this. The system did not get better because the team got better at writing code. The system got better because the system got better at writing code about the system. The locus of improvement shifted one level up. The team is no longer the bottleneck on the team’s own output. The loop is.

This is the difference between linear engineering productivity (more engineers → more output) and compounding engineering productivity (a loop that produces output and improves itself in the same cycle). The first scales with headcount. The second scales with time and discipline.

The Karpathy Loop is one specific instantiation of the broader pattern. The pattern itself is older — Anthropic’s 2025 paper on Building Effective Agents covers the evaluator-optimiser as one of five canonical agent design patterns. What Autoresearch did was prove the pattern works on a real production workload at a real Fortune 500 company, with measurable, board-reportable results. After Shopify, the question stopped being “does this work?” and became “what is your version of this?”

The LangChain Better Harness recipe

The Karpathy Loop is the strategic shape. The LangChain team published the operational recipe.

In April 2026 LangChain published the Better Harness post — a four-step cycle that any AI product team can run, with no exotic infrastructure required. It is the closest thing the industry has to a standard operating procedure for self-improving AI systems. The four steps are simple enough to print on a single page.

Step one — trace mining. Every production session that failed, was abandoned mid-flow, or required human intervention is automatically captured into a trace store. An evaluator agent runs across the traces nightly, clusters them into failure modes, and surfaces the top patterns. This is the input to every other step. If you do not have it, nothing downstream works.

Step two — diagnosis. For each high-volume failure cluster, an evaluator agent (often a stronger model than the production one) generates a hypothesis: the system is failing because the harness is missing instruction X, or because tool Y is being invoked under the wrong condition, or because the context window is being polluted by Z. A human reviews the hypothesis. Most are correct. The ones that are not get sent back for refinement.

Step three — harness edit. The diagnosis becomes a concrete change to the system. Sometimes it is a prompt edit. Sometimes it is a new tool, or a guardrail, or a revised orchestration step. The change is made in the same Git workflow the rest of the engineering team uses. Versioning is non-negotiable — every change is auditable, and every change can be reverted in one command.

Step four — holdout validation. Before the change ships to production, it is validated against a frozen holdout set of representative traces. The dashboard shows before/after metrics. If the change improves the target metric without regressing the guardrails, it ships. If it does not, it is rolled back. No change ships on vibes.

The cadence target is three to five edits per week. World-class teams hit greater than 3.2 per week consistently. This is the Self-Optimisation Rounds metric, and it is the single best leading indicator of whether a self-improving moat is real or theatre.

The recipe is not exotic. It does not require a special model. It does not require a research team. It requires the discipline to set up the trace store, fund the evaluator runs, hold the weekly review, and ship the changes. Most teams do not do this not because they cannot, but because the work does not look like product work in the way their organisation has historically defined product work.

ServiceNow as the master example

The cleanest 2026 case study is ServiceNow, because it shows what happens when three loops compound on each other for several years.

ServiceNow now resolves 90% of its own internal IT requests autonomously. The Now Assist platform processed 95 billion workflow events and over 7 trillion transactions through the context engine in the first quarter of 2026. The product moves a quarter ahead of every competitor in the category, and the gap is widening. The model under the hood is not the explanation — most ServiceNow agents run on the same frontier models the competition has access to.

The explanation is that three loops are running at once and feeding each other.

Loop one — the data flywheel. Every customer running on Now Assist generates trace data that flows back into the platform’s improvement pipeline. With 95 billion workflow events per quarter, the trace volume itself is a moat — no new entrant can match the input distribution.

Loop two — the harness hill-climb. ServiceNow ships harness changes on a weekly cadence across hundreds of agent skills. The changes are versioned, validated, and audited. The cadence is high enough that the system is structurally getting better every week, on every customer.

Loop three — the dogfooding forge. ServiceNow runs ServiceNow internally. Their own IT, HR, and customer-service teams use the same agents the customers use. The 90% self-IT autonomy figure is not a benchmark — it is a daily operational reality for a 25,000-person company. Every internal pain point becomes input to the product team within hours.

The three loops reinforce each other. The data flywheel feeds the harness hill-climb. The harness hill-climb improves the dogfooded experience. The dogfooded experience surfaces failure modes that re-enter the data flywheel. The composite is a system that has been compounding for several years, and the gap to entrants who started compounding two years later is now roughly two years of continuous improvement — measured in tens of thousands of harness edits, the institutional memory of which failure modes have been seen and addressed, and the operational reflexes of a team that has been running the loop long enough to make it routine.

The four properties of a real self-improving system

Pattern matching on “we have a feedback loop” is dangerous, because most product teams do have a feedback loop in some form — they have customer support tickets, they have a roadmap, they have quarterly reviews. None of those is the same thing.

A real self-improving system has four properties, and the absence of any one of them turns the whole into a manual improvement loop with extra steps.

Property one — mechanical learning

The improvement is automated. Humans review the changes, not generate them. A system where every harness edit starts with a human noticing a problem in a meeting, deciding to investigate, finding the trace, writing the fix, and pushing it through review, is not mechanical. It is artisanal. It scales with headcount and dies when senior people leave. A mechanical system has the diagnosis, the candidate fix, and the validation already prepared by the time a human looks at it. The human’s job is to approve, reject, or refine — not to originate.

Property two — holdout validation

Every change is verified against a held-out test set before it ships. Without this, the system optimises for whatever metric is loudest in the trace data, and quality drifts in directions nobody intended. Holdout validation is the difference between a self-improving system and a self-degrading system that thinks it is improving. The holdout set is curated. It is refreshed quarterly. It includes adversarial cases. And it has the authority to block a ship.

Property three — provenance discipline

Every harness edit is versioned in Git, with before/after metrics attached. Reverts are one command away. This sounds basic. It is not done in most organisations. The default state is that the prompt evolves through ten edits over six weeks, no one remembers what the original looked like, and when the metric drops next quarter no one can isolate which edit caused it. Provenance discipline is the difference between a system you can debug and a system you can only re-do.

Property four — compounding velocity as a leading indicator

Self-Optimisation Rounds per week is tracked in the dashboard alongside the customer-facing KPIs. It is not an annual goal. It is reviewed weekly. The team understands that a week with zero rounds is a week the moat did not appreciate. World-class teams hit greater than 3.2 per week consistently, with quarterly averages well above that. Most teams have never measured the metric.

A system with all four properties compounds. A system missing any one of them does not.

The four PM mistakes that kill the moat

These four traps look like self-improving loops from far away. Up close, none of them compounds.

The Self-Correcting Eval Loop

The Karpathy Loop is the strategic shape and the LangChain Better Harness recipe is the weekly cadence. The Self-Correcting Eval Loop is what runs underneath both — the substrate that makes machine-cycle improvement possible without the system optimising itself into an unsafe direction.

It is the autonomous-cadence extension of The Eval Flywheel (AI Evals L3-T24). The basic flywheel runs on a monthly or quarterly cadence with human curation. The self-correcting loop runs on a daily or per-failure cadence with AI curation, and the human role moves to the architectural layer.

Production failures auto-promote to the dataset. A trace that violates a guardrail or scores below threshold is captured, de-duplicated, and added to the eval set within hours. Low-confidence cases — the ones the judge is unsure about — are routed to human review instead of auto-labelled. This single discipline keeps the eval set fresh against a moving input distribution.

Judge calibration auto-runs against a held-out set. The LLM judge is itself evaluated, weekly, against a frozen panel of human-labelled cases. When agreement drops, the judge is recalibrated or the prompt is patched before any new optimisation round runs against it. The judge cannot be the silent point of failure.

CI gates auto-tighten as the suite hardens. Once the system has held a quality bar for four consecutive weeks, the CI gate ratchets up by a small step. Once it has held for a quarter, the gate becomes a contract. The loop’s baseline keeps moving up, never down. This is how the moat appreciates rather than oscillates.

The self-correcting loop is the part of the system that makes autonomous shipping safe. Without it, every “the agent improved itself” story eventually becomes a “the agent regressed in production” story.

Remember this

1. Karpathy’s Autoresearch loop. AI optimises itself at machine cycle time — identify weakness, hypothesise improvement, shadow test, validate, ship, repeat. Humans architect the loop, not each iteration.
2. Shopify’s 53% rendering speedup via 93 autonomous commits. The public proof point. Not exotic agentic capability — a well-defined loop with clear quality gates and ship criteria.
3. The Self-Correcting Eval Loop is the substrate. Production failures auto-promote to dataset. Judge calibration auto-runs. CI gates auto-tighten. Without this, the loop ships regressions confidently.
4. Human role shifts to architecture. PM designs the loop — what counts as success, what is allowed to ship autonomously, what requires human review. Engineering implements. The loop runs.
5. Cycle time is the moat. Machine cycle versus human cycle is the structural advantage. Algorithmic versus discretionary. The gap compounds with each iteration and competitors cannot shortcut it by hiring harder.

In practice

A six-step playbook for moving from human-cycle product work to a machine-cycle self-improving moat. Sequence matters — skipping the substrate is how the loop ships unsafe optimisation.

Step 1 — Build the eval flywheel substrate first. L3-T24 of AI Evals is the prerequisite. Without strong evals, autonomous optimisation has no quality gate and the system optimises against the wrong objective at machine speed.

Step 2 — Map the optimisation surface. Which dimensions can be autonomously optimised — prompt patterns, retrieval reranking, cache management, model selection, tool routing? Which require human decision — architecture changes, vendor switches, pricing? Write the boundary down before the loop ships its first commit.

Step 3 — Design the trust boundaries. What is the loop allowed to ship autonomously? What requires human review? What is permanently out of bounds? The L2-T08 trust architecture and L3-T28 progressive deployment governance apply directly. The boundary is the PM’s deliverable, not engineering’s.

Step 4 — Build the loop’s architecture. Identify-weakness logic (failure clustering on traces). Hypothesise-improvement patterns (evaluator agent, often a stronger model). Shadow-testing infrastructure (replay against historical traffic). Validation gates (holdout set with veto power). Ship criteria (cost, quality, guardrails all green).

Step 5 — Run the loop with weekly architectural review. Engineering and PM review what the loop has shipped, what it is attempting, what is being blocked. The strategic review prevents drift — the loop ships tactics; humans steer architecture. Reviewing every commit defeats the cycle-time advantage; reviewing none defeats the trust boundary.

Step 6 — Translate cycle-time advantage into business language. The board does not care about Self-Optimisation Rounds per week. They care about the structural read: “Our optimisation loop ships roughly 4× more improvements per quarter than competitors at human cycle. Compounding over 4 quarters, that is structural product differentiation that cannot be matched without rebuilding the substrate from zero.” That is the board narrative the moat earns.