Context Economics

The Story

A B2B customer intelligence platform processes 500,000 queries per day for 340 enterprise accounts. Sales teams ask about prospects, deal status, and competitive positioning. The product uses Claude Sonnet for everything -- every query, every intent, every complexity level.

The AI inference bill is $312,000 per month. Growing 15% month-over-month. At current trajectory, the annual run rate will hit $5.4M by Q4. The company's entire engineering budget for the AI product is $8M annually. Inference costs alone are consuming 67% of the product's budget, and the ratio is getting worse.

The PM maps the queries by complexity. 45% are simple lookups that a smaller model could handle. 40% are moderate reasoning tasks. 15% are complex multi-source analyses that genuinely need Sonnet's full capability. One model serves all three tiers. The platform pays Sonnet pricing for a query that asks "What's John's email?"

The Core Idea

Context economics is the discipline of budgeting tokens across CONTEXT layers, understanding cost curves across model tiers, and finding the point where more context costs more than it's worth. At enterprise scale, context engineering IS cost engineering.

Think of it like a restaurant kitchen during a price spike on premium ingredients. A chef who uses wagyu beef for every dish -- burgers, stews, tartare -- will bankrupt the restaurant. Premium models for premium queries, efficient models for routine ones. The dish quality barely changes for the stew. The cost drops by 80%.

Model tiering strategy. The single most impactful cost optimization is not sending every query to the same model. Haiku (or GPT-4o mini, Gemini Flash) handles intent classification, simple lookups, structured extraction -- 40-50% of enterprise queries. Sonnet handles multi-step reasoning and complex analysis -- 40-45%. Opus handles high-stakes decisions where the cost of a wrong answer far exceeds inference cost -- 5-10%.

The Klarna case study. Klarna reported that their AI assistant handled the equivalent of 700 full-time agents' work in its first month. The economics worked because of aggressive context optimization: efficient models for simple tasks, routing only complex disputes to more capable models, and caching frequent query patterns. The headline savings came not from model capability but from the economic architecture around it -- tiering, caching, and routing.

Where This Hits in Production

Cost attribution per tenant. In B2B products, the PM needs to know which tenants are the most expensive to serve. Cost attribution -- tagging every inference call with the tenant ID -- turns "our AI bill is $300K/month" into "Tenant A costs $42K/month for $180K ARR, while Tenant B costs $3K/month for $150K ARR." This data informs pricing and capacity planning.

Caching economics. Anthropic's prompt caching charges $3.75/M for the initial cache write but only $0.30/M for cache reads -- a 90% discount on cached tokens. For a product where the constitution and tool definitions are identical across queries for the same tenant, caching those 3,500 tokens saves 90% of their cost on every subsequent query.

Margin-aware product decisions. "Should we add conversation memory?" becomes "Adding memory increases average context by 1,500 tokens, which costs $547K/year at our volume -- does the quality improvement justify that?"

In Practice: The Three Levers

Three targeted optimizations delivered $729K in annual savings for approximately three engineering weeks of work. 16x ROI in Year 1.

Remember This

1. Every 1,000 tokens saved per query saves $547K/year at 500K queries/day on Sonnet pricing. Context economics is not abstract -- it's a line item that scales linearly with volume and can dominate product budgets within months.

2. Model tiering is the single highest-impact cost optimization. Routing 45% of queries to Haiku saves 60-70% on those queries with negligible quality loss -- because those queries never needed deep reasoning.

3. Optimize for cost per outcome, not cost per query. A cheaper query that requires three follow-ups costs more than an expensive query that resolves in one turn.