- 01.The Inference Treadmill: why fuel costs are dropping 280× while customer bills are rising 320%, and why this paradox is the central economic mechanic of AI products
- 02.The token tsunami: how agentic loops consume 10–30× more tokens than chat-style usage, and why that compounds usage-based bills faster than any pricing-team meeting can react
- 03.The P10/P90 cost spread — why your average user costs $2 and your power user costs $80 — and how that distribution determines whether your unit economics survive
- 04.The cost cliff: the moment your product transitions from low-context to long-context usage patterns and your per-outcome cost jumps 5–10× without a corresponding revenue jump
- 05.The CFO conversation that earns harness investment its budget — translating Inference Treadmill mechanics into gross-margin protection language
The story
Consider the public facts of AI economics in early 2026. The cost per million tokens at the frontier dropped from ~$60 in 2023 to ~$15 in 2024 to ~$2.50 in 2025 to under $1 for many models in 2026 — and to $0.14/M for DeepSeek V4-Flash, with frontier-tier reasoning behind the same number. Fuel cost — the per-token charge from model providers — is down roughly 400× from the early 2023 peak.
Now consider the customer-side reality. Cursor's $7,225 monthly invoice. Replit's −14% gross margin on $252M ARR. Enterprise AI bills routinely 320% higher than the previous year, even at firms that renegotiated their per-token rates downward. The fuel got cheaper. The bills got bigger.
This paradox is the Inference Treadmill. Token prices drop. Model capability grows. Both improvements get consumed by the use case stretching to fill the new capacity. Every model upgrade enables agents to do more — longer contexts, more tool calls, deeper reasoning, multi-step workflows that were infeasible at the old price/capability point. The marginal capability gets consumed by the new use case before any of it shows up as cost reduction.
The Treadmill compounds. As you make the model cheaper, customers do more with it. As they do more, your per-customer infrastructure cost rises. As infrastructure cost rises, the unit economics of your product compresses. The fact that the fuel got cheaper is irrelevant to your P&L — what matters is your cost per successful outcome, and that number is set by how much usage your product enables, not by the price of fuel.
Consider a B2B SaaS team that licenses an AI assistant for $40/user/month. Average usage in the first quarter: 80 sessions per user, 8K tokens per session. At $1/M tokens, the inference cost per user is ~$0.64. Margin is healthy.
Quarter 2 ships a feature: multi-turn agentic workflows. Power users start running 4-hour sessions with deep tool use and long context. Their session token count climbs from 8K to 200K. The team's projected median is 12K tokens per session. The reality is a P10/P90 spread — the median user is at 6K tokens, the P90 user is at 180K tokens. The P90 user costs $5.40/month in inference. Their $40 subscription generates $34.60 in margin contribution. Acceptable.
Quarter 3 ships another feature: persistent agent memory. Power users now run daily multi-hour sessions. The P90 climbs to 500K tokens. Inference cost for the P90 user: $15/month. Margin contribution: $25/month. Still positive but compressing.
Quarter 4 ships the feature the team had been working toward all year: autonomous overnight task execution. Power users let the agent run unattended for 8 hours. Token consumption explodes. P90 climbs to 4M tokens. Inference cost for the P90 user: $120/month. The P90 user is now costing the company $80/month. Their $40 subscription is unprofitable. And there's no way to charge them more without losing them — they're the customers writing case studies and bringing in referrals.
This is the cost cliff. The product transitioned from low-context (chat-style) to long-context (agentic) usage. The unit economics didn't transition with it. The team's pricing model was built on the old assumption. The new assumption requires a different pricing model — outcome-based, tiered usage, abstracted value (see L2-T04 for the pricing taxonomy) — and a different harness architecture (semantic caching, context compression, loop pruning) that drives cost per outcome down at the same quality level.
The CFO sees the gross margin compression in Q4. The team's choice: re-architect the harness, redesign the pricing, or watch the margin keep compressing. The Inference Treadmill demands continuous response. SaaS economics don't.
The core idea
The Inference Treadmill has three mechanics that together break SaaS unit economics:
- The token tsunami. Agentic loops consume 10–30× more tokens than chat-style usage. Multi-turn workflows, deep tool use, long context, persistent memory — each multiplies token consumption. The cost-per-session line grows faster than user count.
- The P10/P90 spread. AI usage is radically heterogeneous across users. The median user might consume 5K tokens/session. The P90 user consumes 100K. The P99 consumes 1M+. The cost distribution is log-normal, not normal — and SaaS pricing models built on per-seat assumptions don't survive log-normal cost distributions.
- The cost cliff. Product transitions from low-context to long-context usage patterns produce step-function cost increases. The team that priced the product for chat usage doesn't survive the agentic transition. The fix isn't pricing alone — it's harness re-engineering plus pricing redesign together.
The token-price drops do not save you. They get consumed by the use case stretching to fill the new capacity. The ~400× fuel-price drop produced a 320% customer-bill increase in the same period because every cheaper token enabled three more tokens of usage.
The Inference Treadmill is the structural mechanic in AI economics where token-price drops and capability gains are continuously consumed by use cases stretching to fill the new capacity. The result: declining unit economics for products whose pricing assumed the previous capability level. Three components — the token tsunami, the P10/P90 spread, the cost cliff — together explain why fuel costs are dropping ~400× while customer bills are rising 320%. The mechanic is permanent. The response is harness re-engineering plus pricing redesign together, on a continuous cadence.
The definitionHighway expansion. Adding lanes to a congested highway doesn't reduce congestion — it induces additional driving until the new capacity fills. Traffic engineers call this induced demand. The Inference Treadmill is induced demand applied to AI: cheaper tokens induce more agentic usage; more capable models induce longer contexts; both improvements get consumed by the use case before any of it shows up as cost reduction in the customer's bill or your gross margin. The fix isn't more lanes — it's smarter routing, better signaling, and pricing that scales with congestion. Same logic applies to AI products.
Think of it like:The concept — visualized
The token tsunami
A chat-style interaction is bounded: user types a message, model responds, user types again. Tokens per session: 1K–10K typical.
An agentic loop is unbounded by design. The agent reads context, calls tools, processes tool responses, plans next steps, calls more tools, recurses. A 10-step agent with 10K tokens of context per step consumes 100K+ tokens before producing the final output. A multi-hour autonomous session consumes millions.
Anthropic's harness design article documented this directly: long-running agentic sessions can consume 50–100× more tokens than the equivalent chat-style task because the agent's planning, reflection, and tool-call rounds each cost tokens. The team that prices for chat usage and ships agentic features is not pricing for what they shipped.
The fix isn't to ban agentic usage. It's to measure cost per outcome and design the harness to keep that number flat as agentic capability grows. The patterns from Eval Economics (AI Evals L3-T26) — context compression, semantic caching, loop pruning, model tiering — are the toolkit. Without them, the token tsunami compresses margin every quarter.
The P10/P90 spread
SaaS pricing models assume usage roughly clusters around a mean. Per-seat pricing works when the cost-per-seat is roughly stable. AI breaks this assumption. Usage distributions are log-normal — the P90 user costs 10–40× the P50 user.
What this means in practice: your average-user dashboard tells you nothing about your unit economics. A team reporting "average inference cost is $2/user" is hiding a P90 of $80/user. If your subscription is $40, the P90 is unprofitable, and the P90 is the cohort that's most engaged and most influential.
The fix is to measure and price for the distribution, not the mean. Three patterns:
- Tiered usage caps — a base tier with usage limits, overage pricing for high-consumption users, premium tier for power users. Aligns price with cost.
- Outcome-based pricing — pay per successful outcome rather than per-seat. Decouples price from token consumption. (See L2-T09 for the outcome-based pricing playbook.)
- Abstracted value pricing — package the AI capability into a higher-level unit (workflow completion, document processed, deal closed) and price the unit, hiding the token economics from the customer. (See L2-T04 for the pricing taxonomy.)
A team without one of these patterns is running per-seat pricing on log-normal cost distributions. The math doesn't work. The margin compression is a matter of when, not whether.
The cost cliff
Product transitions from low-context to long-context usage produce step-function cost increases. The Q4 story above is the mechanic. The team didn't gradually grow into the cost cliff — they shipped a feature that flipped the usage pattern, and the cost jumped 5–10× in a single quarter.
The cost cliff is predictable but only if the team is watching. The signature: a product roadmap that adds agentic capability (autonomous task execution, multi-hour sessions, persistent memory, deep tool use) without a corresponding pricing redesign. The roadmap is enabling the cost cliff. The pricing model isn't catching up.
The fix is to gate every agentic feature behind a unit-economics check. Before shipping autonomous overnight execution, model the projected cost per outcome at the new usage pattern. If it breaks the unit economics, either redesign the pricing first or redesign the harness to keep cost per outcome flat.
DeepSeek V4-Flash at $0.14 / $0.28 per M tokens — and V4-Pro at $1.74 / $3.48, roughly 4.5× under Claude Opus 4.7 on the Artificial Analysis Intelligence Index — is the 2026 lever. Routing the routine 80% to V4-Flash and reserving frontier capability for the hard 10–20% can flatten the cost cliff without flattening the user experience. The L3-T03 chapter on multi-model orchestration is the operational discipline.
Where this hits in production
The Cursor case study is the public reference for the Treadmill. The team shipped agentic features faster than the pricing model adapted. The $7,225 invoice was a P99 outlier that became visible because it was extreme. The P90 reality was less visible but more dangerous — a quietly unprofitable cohort whose existence wasn't reflected in any dashboard. The fix was harness re-engineering plus pricing redesign together. The lesson: the pricing model and the harness architecture have to evolve in lockstep with the product roadmap. Ship a feature, redesign the pricing. Ship a feature, redesign the harness. Skip either step and the Treadmill compresses margin.
The DeepSeek V4 pricing pattern reframes the model decision. V4-Flash holds the $0.14/M floor that V3.2 set a year ago — but with frontier-tier reasoning included, and with cache hits at $0.028/M (an 80% discount on repeated context, which is where most agent traffic actually lives). For most enterprise use cases, the savings are large enough to absorb significant token-tsunami growth. The team that routes routine work to V4-Flash and reserves frontier capability for the hard 10–20% extends the runway on the Treadmill considerably.
The Replit −14% margin story is the cost-cliff in public. $252M ARR sounds healthy. Negative gross margin means every dollar of revenue costs more than a dollar to produce. The fix isn't sales motion — it's fundamental architecture. Either re-engineer the harness to drive cost per outcome down (Cursor's response) or redesign the pricing to align with the cost distribution (the L2-T04 taxonomy). Skip both, and the company scales itself into bankruptcy.
The trap
Trap 1: Watching average cost while ignoring P90. Average dashboards hide the cost distribution. The team reports $2/user average. The P90 is $80. The team is hemorrhaging margin on the cohort that matters most. The fix: dashboard the P10, P50, P90, P99 separately. The shape of the distribution is the story.
Trap 2: Trusting that token prices will solve unit economics. Every quarter, token prices drop. Every quarter, customer usage grows. The two effects cancel. The team that's "waiting for token prices to fix the margin" is on the Treadmill — running in place. The fix is to assume token prices will keep dropping and customer usage will keep growing, and design the unit economics for the equilibrium.
Trap 3: Shipping agentic features without pricing redesign. Autonomous task execution, multi-hour sessions, persistent memory, deep tool use — each shifts the usage pattern and triggers the cost cliff. Shipping these features under per-seat pricing is the textbook setup for negative gross margin. The fix: pricing redesign and harness re-engineering ship together with the agentic feature, not after the CFO complains.
Trap 4: Treating the Treadmill as a one-time event. It's not. The Treadmill runs continuously. Each model upgrade triggers a new round. Each customer-pattern shift triggers a new round. The discipline is to expect a Treadmill response cycle every quarter, build it into the operating model, and resource the team to do it.
Remember this
- Fuel costs are dropping ~400×. Customer bills are rising 320%. That paradox is the Inference Treadmill — and DeepSeek V4 just widened both sides of it.
- Three mechanics: the token tsunami (agentic loops consume 10–30× more tokens), the P10/P90 spread (log-normal cost distributions break per-seat pricing), the cost cliff (product transitions trigger step-function cost increases).
- The token-price drops do not save you. They get consumed by the use case stretching. Plan for unit economics at the equilibrium, not the current price point.
- Watch P90 cost, not average. Average hides the cohort that matters. Log-normal distributions need log-normal pricing models — tiered, outcome-based, or abstracted-value.
- Ship pricing redesign and harness re-engineering with every agentic feature. Skip either step and the Treadmill compresses margin every quarter.
In practice
Step 1: Build the cost-distribution dashboard. Replace "average cost per user" with P10, P50, P90, P99. Update weekly. The shape of the distribution is the early-warning indicator for the cost cliff.
Step 2: Pair every agentic feature on the roadmap with a unit-economics projection. Before shipping autonomous task execution, persistent memory, multi-hour sessions, project the P90 cost per outcome at the new usage pattern. If it breaks the unit economics, redesign before ship.
Step 3: Implement model tiering. Route low-complexity work to a cheap model (DeepSeek V4-Flash, Llama 4, Phi-5). Reserve frontier capability for the hard 10–20%. The L3-T03 chapter on multi-model orchestration is the playbook.
Step 4: Add the harness-cost levers to the engineering roadmap. Context compression, semantic caching (90% reduction on cacheable queries), loop pruning (50–70% reduction on agentic loops), continuous batching. These are the L2-T05 Inference FinOps mechanics. Each lever buys runway on the Treadmill.
Step 5: Move pricing from per-seat toward outcome-based or abstracted-value. The L2-T04 pricing taxonomy is the operational guide. Per-seat pricing on log-normal cost distributions is a structural mismatch. The transition is a 2–4 quarter project, not a single launch.
Step 6: Translate the Treadmill mechanics into CFO language. "Fuel costs dropped 50% but customer usage grew 4×; net cost per outcome is up 100%. Without harness re-engineering and pricing redesign, gross margin compresses 8–12 points per quarter. Recommendation: invest $X in harness work and $Y in pricing transition over the next 2 quarters to hold gross margin flat against the current trajectory." The translation earns the budget.
The practice — visualized
