Prompt ≠ Context

The Story

In June 2025, Tobi Lütke — the CEO of Shopify, a company that handles billions of dollars of e-commerce every quarter — posted a sentence on X that quietly changed the conversation. He wrote that the most important skill in AI is not prompting. It is context engineering: "the art of providing all the context necessary so that an LLM can work out a reasonable solution to the task at hand."

(An LLM — Large Language Model — is the AI that produces text. Think of it as the engine. The context is the fuel.)

The line landed because it named something practitioners had been feeling for two years and could not put words to. Teams had poured millions into prompt optimisation — rewriting instructions, A/B-testing tone, tuning settings called temperature (a knob that controls how random the model's word choice is — turn it down for predictable answers, up for creative ones). The instructions got sharper. The model got smarter. The product still misfired.

Consider a B2B customer-support platform serving enterprise accounts. The team spends three months perfecting prompts for their AI response assistant. They test hundreds of phrasings. They add guardrails to the instructions. They tune temperature. The prompts are excellent.

Then a Tier-1 client reports that the AI keeps recommending a discontinued product. The prompt does not mention any product — it is a general instruction to be helpful and accurate. The real problem lives somewhere else: the system that fetches background information for the AI is pulling from an outdated product catalog. The knowledge layer is stale. No amount of prompt rewriting will fix a problem that lives outside the prompt.

The team was optimising the 5% they could see. The failure lived in the 95% they weren't thinking about.

Source: Tobi Lütke on X.

The Core Idea

Every AI product has an invisible architecture that decides how it behaves. Users see a clean interface — they type a question, the AI answers. It looks like a conversation between a person and a smart machine.

It is not.

What's actually happening is that the model receives a single, large block of text — assembled from many sources, in a specific order, formatted in a specific way — and produces its answer based on everything in that block. The user's message is one small piece of this assembled text. Often less than 5% of it.

That assembled block is the context. The difference between an AI product that works reliably and one that fails unpredictably almost always lives in the context — not in the user's message.

Prompt engineering became a buzzword in 2023. It named the craft of writing better instructions — clearer phrasing, sharper constraints, a few examples pasted in. Real work. Genuinely useful. Fundamentally limited.

Limited because a prompt is one ingredient inside the context. It is the instruction sheet you hand to a chef. But what about the ingredients (background information the AI looks up at runtime), the kitchen setup (the tools the AI can use to take action), the customer's order history (memory of previous interactions), the health regulations posted on the wall (the system rules that constrain behaviour), and the plating template (the exact shape the answer must come in)?

You can write a perfect instruction sheet. If the ingredients are stale, the dish fails.

(Inference time — the moment the model produces an answer. Not training. Not preparation. The instant the input gets turned into output.)

Think of it like: a prompt is the question on an exam. Context is the entire exam packet — the instructions stapled to the front, the formula sheet they handed out, the reference book on the side, the scratch paper, and the question itself. The student's answer depends on all of it. Not just the question.

Andrej Karpathy — one of the original researchers behind modern AI systems and the former AI lead at Tesla — described the shift this way: building with LLMs is fundamentally different from traditional software because the "program" (the prompt and context) and the "runtime" (the model) are entangled in ways that traditional debugging tools don't handle well. Translation: in normal software, the code is one thing and the input is another, and you can fix the code without touching the input. In AI, everything the model sees in a single moment shapes everything it produces in the next. The input IS the program.

Anthropic — the company behind the Claude model — crystallised this in their engineering blog on building effective agents. They described context as a layered system: the system prompt defines the agent's behaviour and available tools; retrieved documents provide domain knowledge; conversation history maintains state; tool definitions expand capabilities. Not one prompt. An engineered information environment.

The practical consequence is a shift in where the PM's attention has to go. Traditional product work focused on features, flows, and screens. AI product work adds a new surface: the information environment. What does the model see when it runs? How is that information assembled? What happens when a piece is missing, stale, or self-contradictory?

Anthropic put it more bluntly in the same blog: "The most common source of failure in AI applications isn't model capability — it's context quality. The model is doing exactly what the context tells it to do. If the response is wrong, the first question should be: what's wrong with the context?"

That reframe — from the AI is broken to what did we feed the AI — is the mental shift that separates teams who iterate quickly from teams who keep swapping models hoping the next one will magically understand their domain.

The history helps practitioners place what they already know:

Prompt engineering (2023) sharpened the instruction text. Real work. Still useful. Limited to what you can type into a text field.

RAG (2024) — Retrieval-Augmented Generation, the pattern of fetching relevant documents from a knowledge base and dropping them into the AI's input — added live knowledge. The AI could now "know" things beyond its training data. Important. Also one ingredient inside the larger meal.

Context engineering (2025+) recognises that instructions and knowledge are two of many ingredients fighting for space inside the context window — the maximum amount of text the model can hold in mind at once, like the maximum number of pages a person can keep open on their desk. Memory, tools, execution results, output structure, examples — all of these have to be orchestrated together. Context engineering is the complete picture.

This is not a replacement hierarchy. It is accumulation. If you know prompt engineering, you've mastered the innermost layer. If you know RAG, you've added the knowledge layer. Context engineering is the discipline of running every layer at once.

Where This Hits in Production

The invisible cost of prompt-only thinking. Enterprise teams that organise around prompts build prompt libraries — Notion databases or Git repositories full of tested instructions. The libraries feel productive. They create a false sense of completeness. A prompt works in the context it was tested in. Move it to a different product, a different retrieval setup, or a different model version, and it breaks in ways the prompt itself cannot explain. The library is a collection of recipes without ingredients.

Cross-functional ownership gaps. When AI quality is framed as prompt quality, ownership defaults to whoever writes the prompts — often a junior engineer or a copywriter. When AI quality gets reframed as context quality, ownership expands to include the retrieval engineer (knowledge layer), the platform engineer (tool definitions), the data team (memory systems), and the product manager (what context to assemble, in what priority, for what user). The reframe redraws the org chart of who is responsible for what.

Multi-model portability. Prompts are model-specific — phrasing that works on GPT-4o sometimes underperforms on Claude. Context pipelines — the assembly logic that loads instructions, retrieves knowledge, injects memory, defines tools, applies templates — are far more portable. The pipeline survives a model swap. The prompt library doesn't. In a market where the frontier model changes every six months, this is the difference between a team that adapts in a day and a team that rebuilds for a quarter.

Connecting the Dots

There's a pattern that keeps showing up across every series in this corpus: the thing practitioners assume is the whole picture turns out to be a small piece of a much larger system. In evals, public benchmarks felt like the complete quality picture — until you noticed they measure the model, not your product. Here, prompts feel like the complete input — until you notice they're 5% of what the model actually processes.

Both realisations point at the same lesson: the PM's job is not to optimise the visible piece. It is to design the system that's invisible to the user.

On the autonomy spectrum. Across this series I'll keep referring to a range — from simple AI features at Level 1 (autocomplete, smart search) to fully autonomous agents at Level 7 (a system that handles a multi-step task end-to-end with no human in the loop). The full spectrum lives in Spectrum of Agency (Agentic Stack L3-T22). For this post, the relevant fact is that the prompt-to-context shift gets more urgent at every level. At Levels 1–3 (autocomplete, chatbot, assistant), the context is small — a system prompt, maybe some retrieved docs, a bit of conversation history. A skilled prompt engineer can manage it. At Levels 5–7 (agents, autonomous systems), the context becomes dynamic, multi-source, and constantly changing — the agent is selecting its own tools, managing its own memory, deciding what to retrieve at every step. You cannot prompt-engineer your way through a 10-step agent chain where every step assembles different context.

The moat angle. Context pipelines are harder to copy than prompts. A prompt can be reverse-engineered from a product's output in minutes — paste the output into an LLM and ask it to guess the instructions. The context assembly system — what gets retrieved, how it's ranked, what memory is loaded, how tools are prioritised, what template is applied — represents months of iterative product decisions. That's institutional knowledge baked into infrastructure, not a text string a competitor can copy. The deeper reason model-quality alone stopped being a moat in 2026 is here: the moat moved one layer up.

Remember This

1. A prompt is what the user types — maybe 50 tokens. Context is everything the model sees at inference time — instructions, examples, knowledge, memory, tools, results, structure. In production, the prompt is less than 5% of the context.

2. When the AI gives a bad answer, the first question is not is the model good enough? It is what's in the context? Most production failures are context failures misdiagnosed as model failures.

3. Context pipelines travel across models. Prompt libraries don't. Build the pipeline.

In Practice: Seeing the Real Context

Anthropic's Claude Code runs as a command-line tool. A developer types fix the authentication bug in the login module — twelve words. The model receives 14,200 tokens.

The capitalised labels you'll see below — CONSTITUTION, KNOWLEDGE, TRACKS, EQUIPMENT, TEMPLATE — are five of the seven letters of a framework I call the CONTEXT stack: the seven kinds of information every agent assembles before producing a single word. Each layer answers one question:

• C — Constitution. Who am I and what are my rules?
• O — Observations. What good and bad answers have I been shown?
• N — kNowledge. What facts do I need to look up right now?
• T — Tracks. What do I remember from before?
• E — Equipment. What tools can I use?
• X — eXecution. What just happened when I ran a tool?
• T — Template. What shape must the answer come in?

The full unpack lives in The CONTEXT Stack (Agentic Stack L1-T03). For now, the visual below is the discovery: the user typed 12 words. The system handed the model 14,200.

(Tokens — the chunks of text the model processes, usually a few letters, sometimes a whole word. A page of normal English is roughly 500–700 tokens.)

The user typed 12 words. The system assembled 14,200 tokens of context before the model produced a single response token. The Constitution alone — the master instruction set that defines how Claude Code behaves — is 3,800 tokens. The tool definitions are 2,600 tokens. The retrieved project files are 4,200 tokens.

The user's message is less than 1% of what the model sees.

That is what context engineering means in practice. Not the 50-token prompt. The 14,200-token information environment that determines whether the model answers usefully or hallucinates a fix that breaks the codebase. When the response is wrong — when the AI suggests editing the wrong file, uses a function that was deprecated last quarter, or ignores a project convention everyone on the team understands — the diagnosis almost always points at the context: the wrong file got retrieved, the project instructions were stale, a tool definition was ambiguous, the memory layer didn't carry forward what mattered.

The PM who thinks in prompts asks: how do I phrase this request better? The PM who thinks in context asks: what information is the model missing, and how does my system supply it?

That single question — what is the model missing? — is the diagnostic move that turns a struggling AI product into a working one.

Up Next — The Context Window Itself

Now that you can see context exists, the next question is uncomfortable: how big can it actually be? Every model has a hard ceiling on how much text it can hold in mind at once — a number called the context window. Some models hold 8,000 tokens. Some hold 200,000. A few hold a million. The number sounds like a technical specification. It is actually a product constraint with consequences for cost, reliability, and how much memory your system can carry.

When the context window is tight, every layer competes for space — and the layer that loses determines what your AI gets wrong. When the context window is huge, the temptation is to dump everything in and hope the model figures it out. Both moves are mistakes.

The next post is about the math of the context window and the engineering decisions that math forces. We'll work through real numbers from production systems and show why "more context" is not the same as "better context."