My AI Learnings — The AI Production Chasm

THE AIPRODUCTIONPRODUCTIONCHASM

Most AI products fail not because the model is wrong, but because everything around the model is missing. These working notes map the gap between demo and production.

PrototypeThe ChasmScaled Production
From the author

I started this work because the same patterns kept showing up under different vendor logos.

  • —Salesforce. 5,000 paid Agentforce deals. 150,000 enterprise customers.
  • —Microsoft. 3.3% paid Copilot penetration across 400M+ M365 seats.
  • —Klarna. Fired 700 support agents in 2024. Quietly rehired in 2025.
  • —Anthropic. Same Claude Opus. Better harness. 2× production task completion.

It opens with My Point of View — Deep Research on Frontier Companies, where I read the market instead of the craft: signal cut from noise across the seven companies pacing the AI economy. Then come the four craft series that document what I learned, in the order the questions arrived. Some firsthand from production. Some synthesised from deep reading. Some I’m still uncovering. Where I am, I say so. This is the working notebook of a PM trying to operate at the level of the 0.1%, in public.

The Career Arc

FOUR SERIES.
ONE CAREER ARC.

Read them in the order the questions arrive in a real PM career. Each series earns the next — and the shortcut is the long way around.

Series 1 // Foundation

Agentic Stack — Context Designer

Before agents, before harnesses, before any of the exciting stuff — the model only sees what you put in the window. 30 topics that build the vocabulary every PM in 2026 needs to think clearly about AI systems. Master this and every upstream problem gets easier.

Series 2 // Build

Harness Engineering — System Architect

The model is a component. The harness is the system — the part that decides what context gets assembled, when to degrade gracefully, and whether you can afford any of it. 8 articles on the production discipline that decides whether your agent ships. The engineering layer where models stop being demos and start being products.

Series 3 // Prove

AI Evals — Quality Owner

A probabilistic system you cannot measure is a probabilistic system you cannot ship. 30 topics + 5 leading-edge questions on the discipline of measuring non-determinism honestly. The eval is the product requirement — a PRD without an eval describes nothing executable.

Series 4 // Monetise

AI PM OS — Strategist + Operator

Building it well does not mean it makes money. That is a different problem. 30 main posts + 5 strategic-decision companions on the layer above engineering — product-market fit, defensible pricing, real ROI, and a board narrative that earns the next round of investment. The operating system of the 5%.

Vocabulary, then craft, then proof, then business. Skip a layer and the next one has nothing to stand on — which is how most of the anti-patterns below get written.

The Traps

What goes wrong

Anti-Pattern

"We'll use the best model"

Teams pick GPT-5.5 or Claude 4.7 Opus and assume quality follows. The model is 10% of the product. The harness is 90%.

Anti-Pattern

"Let's add an eval later"

Building blindly without a ground-truth dataset. If you can't measure the baseline, you can't improve it.

Anti-Pattern

"The prompt just needs tweaking"

Trying to solve architectural data-retrieval problems by writing longer, more complex mega-prompts.

Anti-Pattern

"Ship fast, fix accuracy later"

Deploying AI to users without guardrails, destroying user trust permanently on the very first hallucination.

MY POINT OF VIEW // FIELD INTELLIGENCE

FRONTIERCOMPANIES

My point of view on the seven companies setting the pace of the AI economy — audited profit vs run-rate, the silicon and open-weight races, and where AI value is actually created and captured. The market read that frames everything else here.

WORKING NOTE // SERIES

AGENTICENGINEERING STACK

The systems thinking of AI agents grounded on context engineering principles. How to shape the information an agent sees — because what it doesn't know, it invents.

WORKING NOTE // SERIES

HARNESSENGINEERING

The definitive guide to building deterministic wrappers around non-deterministic models. Designing safety boundaries that survive contact with enterprise reality.

WORKING NOTE // SERIES

AI EVALS &OBSERVABILITY

A practitioner's guide to measuring what actually matters. The absolute discipline of knowing whether your AI actually works in production, backed by ground truth.

WORKING NOTE // SERIES

AI PMOPERATING SYSTEM

The strategy + execution layer above the engineering. 30 main posts plus 5 BONUS companions on shipping AI products that work AND make money — the Magnifying Glass thesis, vendor U-turns, the SaaSpocalypse pivot, and the operating disciplines of the 5%.

"These are the reps.
Not polished theory, but the late-night notes you write when reality breaks your assumptions."