My AI Learnings: The AI Production Chasm

THE AIPRODUCTIONPRODUCTIONCHASM

Most AI products don't fail because the model is wrong. They fail because everything around the model is missing. These are working notes on the gap between demo and production.

5 series 100+ topics 1 point of view updated 2026
PrototypeThe ChasmScaled Production
From the author

I started this work because the same patterns kept showing up under different vendor logos.

It opens with My Point of View: Deep Research on Frontier Companies, where I read the market instead of the craft. Signal cut from noise across the seven companies pacing the AI economy. Then come the five craft series, in the order the questions actually arrived. Some firsthand from production. Some synthesised from deep reading. Some I’m still figuring out. Where I am, I say so. This is the working notebook of a PM trying to operate at the level of the 0.1%, in public.

The Career Arc

FIVE SERIES.
ONE CAREER ARC.

Read them in the order the questions actually arrive in a PM career. Each series earns the next. The shortcut is the long way around.

Series 1 // Foundation

Agentic Stack, Context Designer

Before agents, before harnesses, before any of the exciting stuff, the model only sees what you put in the window. 30 topics that build the vocabulary every PM in 2026 needs to think clearly about AI systems. Master this and every upstream problem gets easier.

Series 2 // Build

Harness Engineering, System Architect

The model is a component. The harness is the system: the part that decides what context gets assembled, when to degrade gracefully, and whether you can afford any of it. 8 articles on the production discipline that decides whether your agent ships. The engineering layer where models stop being demos and start being products.

Series 3 // Operate

Environment Engineering, Boundary Owner

The harness decides what may be attempted. The environment decides what that attempt can reach, whose authority it carries, what survives it, and how fast you can undo it. 8 decisions that are usually inherited from a prototype instead of chosen. The environment is the product's consequence boundary.

Series 4 // Prove

AI Evals, Quality Owner

A probabilistic system you can't measure is a probabilistic system you can't ship. 30 topics and 5 leading-edge questions on the discipline of measuring non-determinism honestly. The eval is the product requirement. A PRD without an eval describes nothing executable.

Series 5 // Monetise

AI PM OS, Strategist + Operator

Building it well doesn't mean it makes money. That's a different problem. 30 main posts plus 5 strategic-decision companions on the layer above engineering: product-market fit, defensible pricing, real ROI, and a board narrative that earns the next round of investment. The operating system of the 5%.

Words, craft, proof, business. Build in order, or the tower falls. Every anti-pattern below skipped a floor.

The Traps

What goes wrong

Anti-Pattern

"We'll use the best model"

Teams pick GPT-5.5 or Claude 4.7 Opus and assume quality follows. The model is 10% of the product. The harness is the other 90%.

Anti-Pattern

"Let's add an eval later"

Building blind, no ground-truth dataset in sight. If you can't measure the baseline, you can't tell whether the next change helped or hurt.

Anti-Pattern

"The prompt just needs tweaking"

Trying to fix an architecture problem (retrieval, memory, tool choice) by writing a longer, more elaborate mega-prompt. It never works.

Anti-Pattern

"Ship fast, fix accuracy later"

Shipping AI to users with no guardrails. One bad hallucination and the trust is gone for good.

MY POINT OF VIEW // FIELD INTELLIGENCE

FRONTIERCOMPANIES

My read on the seven companies setting the pace of the AI economy. Audited profit against run-rate, the silicon and open-weight races, and where AI value is actually created and captured. The market view that frames everything else here.

WORKING NOTE // SERIES

AGENTICENGINEERING STACK

The systems view of AI agents, grounded in context engineering. How to shape what an agent sees, because what it doesn't see, it invents.

WORKING NOTE // SERIES

HARNESSENGINEERING

How to build deterministic wrappers around non-deterministic models. Safety boundaries that survive contact with enterprise reality, and the trade-offs you make to get there.

WORKING NOTE // SERIES

ENVIRONMENTENGINEERING

What world have we placed around the agent? Reach, identity, containment, persistence, recovery, realism, ownership — the eight decisions that determine what a single run can cost you, and who answers for it.

WORKING NOTE // SERIES

AI EVALS &OBSERVABILITY

A practitioner's take on measuring what actually matters. How to know your AI works in production, backed by ground truth, not vibes.

WORKING NOTE // SERIES

AI PMOPERATING SYSTEM

The strategy and execution layer above the engineering. 30 main posts plus 5 BONUS companions on shipping AI products that work and make money. The Magnifying Glass thesis, vendor U-turns, the SaaSpocalypse pivot, and the operating habits of the 5%.

"These are the reps.
Not polished theory, but the late-night notes you write when reality breaks your assumptions."