When AI Is the Right Answer

The story

Consider a financial services PM whose VP of Engineering proposes an "AI-powered transaction categorization" project. The team estimates 6 months and $1.2M to fine-tune a model on the company's transaction data. Leadership is excited. The PM, who has read this chapter, asks one question: "What does the rules-based version of this look like?"

The Tech Lead resists the question — "AI is the right answer here" — but agrees to a 1-week experiment. A junior engineer writes a 200-line rules engine that handles 89% of transactions correctly. The remaining 11% — genuinely ambiguous cases — get routed to a single LLM API call (~$0.003/transaction). Total cost: 1 week and $400/month at projected volume. The 6-month, $1.2M fine-tuning project is canceled. The product ships in 14 days.

This is Boring AI. The most expensive PM mistake in 2026 is building bespoke when off-the-shelf would have shipped in two weeks. The bias toward sophisticated AI solutions — the same bias that makes consultants rich and CFOs bankrupt — is the bias the discipline of this chapter exists to counter.

Consider a second team that's three months into a fine-tuning project for legal document review. They've collected 50,000 labeled examples. They've trained a custom model. The accuracy is 84%. The deployment cost is $40K/month. The team is proud of the work.

A consultant looks at the output and asks: "Why isn't this RAG?" The team protests — "The legal domain is too specialized for a general model." The consultant runs an experiment: he feeds the same documents through a general-purpose API plus a retrieval layer pulled from the firm's own document corpus. Accuracy: 87%. Deployment cost: $4K/month. The 90% accuracy gain came from retrieving the right context, not from fine-tuning. The custom model becomes a 6-month write-off.

A 2025 industry survey reported that 80% of enterprise AI use cases solve with RAG before fine-tuning is even worth considering. The fine-tune-first instinct is one of the most expensive habits in AI PM, and it persists because fine-tuning sounds more impressive in the all-hands deck.

Consider a third team that builds a bespoke AI feature for customer support — a 4-month project to ship a custom assistant. Two weeks after launch, OpenAI ships an Assistant API update that does 95% of what the team built, with better quality, at 1/10 the cost. The team's 4-month project is now a maintenance liability. The lesson isn't "AI moves fast." The lesson is the boundary between "buy off-the-shelf" and "build bespoke" should be drawn with the assumption that the off-the-shelf surface is going to keep expanding. Build bespoke only where the bespoke layer is durable — domain-specific knowledge, proprietary workflow, integration depth, harness mastery from L1-T01.

Three teams, three versions of the same lesson: the cheapest AI is the AI you didn't build, and the second cheapest is the off-the-shelf API call you bought instead. The Boring AI discipline is the diagnostic that lets a PM walk into a roadmap review and tell, in 30 seconds, whether the proposed initiative is starting at the right rung of the ladder.

The core idea

The AI investment ladder has four rungs:

The Boring AI discipline says: start at rung 1, validate, step up only when you have evidence-grade proof that the next rung delivers >5–10% improvement on the metric that actually matters. The bias in most teams is to start at rung 3 or 4. The bias is wrong.

The concept — visualized

The four rungs in detail

Rung 1: Rules. A 200-line rules engine handles 70–95% of many "AI" problems at nearly zero cost. Rules are deterministic, debuggable, and don't drift. The trap is that rules don't make for good demos. The fix is to measure: if rules can solve >85% of cases with confidence, the AI version of this problem starts at the LLM-fallback layer, not at fine-tuning. The financial-services categorization story is the canonical pattern.

Rung 2: APIs. A single API call to a frontier model handles a large class of problems that rules can't (open-ended generation, ambiguous classification, multi-step reasoning). Cost is per-token, variable, transparent. No infrastructure. No training. No maintenance. Most B2C and consumer use cases never need to leave this rung. The trap is treating "we use the API" as commodity — the differentiation lives in prompt design, retrieval, eval discipline, and harness work, not in custom training.

Rung 3: RAG. Adds a retrieval layer that grounds the API call in your proprietary data. Requires infrastructure (vector database, embedding pipeline, retrieval logic) and ongoing maintenance (data freshness, relevance tuning). The defensibility upgrade comes from the data, not the model. 80% of enterprise use cases live here. The trap is treating RAG as a one-time setup rather than a living system. RAG that isn't continuously evaluated and refreshed degrades quietly. The connection to Evaluating RAG (AI Evals L2-T14) is the eval discipline that keeps RAG honest.

Rung 4: Fine-tuning. Train a custom model on your data. Highest cost, highest infrastructure commitment, highest ongoing maintenance. Only worth it when: (a) RAG can't achieve the required quality threshold, (b) the volume justifies the per-token cost savings versus API + RAG, (c) the use case is durable enough that the fine-tuned model won't be obsolete in 6 months when the next frontier API update lands. The trap is fine-tuning because it sounds impressive. The fix is to require evidence-grade proof — A/B comparison against RAG on the actual production traffic — before approving the fine-tuning budget.

The 5–10% ROI threshold

Stepping up the ladder costs real money. Each rung adds infrastructure, maintenance, and engineering time. The cost of stepping up is justified only if the ROI on the next rung exceeds 5–10% on the metric that actually matters.

What "the metric that actually matters" is depends on the use case. For customer support: cost per resolved ticket. For document review: time per document plus accuracy. For sales assistance: revenue per assisted deal. The metric must be a P&L line item, not a satisfaction score.

The 5% threshold is the lower bound for cases where the engineering cost is small. The 10% threshold is the lower bound for cases where the engineering cost is significant. Below those thresholds, the next rung's complexity isn't paying for itself — and the team is investing in technical sophistication rather than business outcome.

The discipline: before approving any rung-up investment, the proposing team must run an evidence-grade comparison (A/B test, holdout cohort, or controlled simulation) on real production traffic and demonstrate the ROI threshold is met. The friction of the requirement is the feature — it filters out rung-up proposals that wouldn't survive contact with the data.

Where this hits in production

The DeepSeek V3.2 pricing pattern reframes the API-vs-fine-tuning conversation. At $0.14/M tokens, DeepSeek V3.2 is 95% cheaper than the frontier on many use cases. For teams that were considering fine-tuning to reduce per-token cost, DeepSeek (or comparable mid-tier models) often delivers the cost reduction without the fine-tuning commitment. The model layer is becoming a commodity, which makes the API rung viable for use cases that historically forced fine-tuning. The L1-T01 lesson — model is commodity, harness is moat — applies directly: the team's leverage shifts from custom training to harness work.

The OpenAI Assistant API pattern is the buy-vs-build reset. Every six months, the off-the-shelf API surface expands. Assistants, Computer Use, Code Interpreter, file search, retrieval as a managed service. Teams that built these capabilities bespoke in 2023 are now maintaining infrastructure that the API ships better and cheaper. The discipline: build bespoke only where the bespoke layer is durable — domain knowledge, proprietary workflow, integration depth, harness mastery. Build commodity, and the next API update obsoletes the work.

Klarna's customer service initiative is a Boring-AI diagnostic. The 2024 announcement framed it as a fine-tuned bespoke system. The 2025 reality (re-hiring humans for nuance) suggests the Boring-AI hierarchy was inverted. A rules-and-RAG hybrid for the volume work plus humans for nuance might have shipped faster, cost less, and avoided the headline reversal. The lesson isn't that Klarna's AI was bad. The lesson is that the investment level didn't match the durability of the use case. Volume support is rules + RAG territory. Nuance support is human territory. Fine-tuning for a use case that will be reorganized 18 months later is the highest-cost path to a 14-month-old answer.

The trap

Trap 1: The "AI is the right answer" framing. When the question is "how do we use AI for this?" instead of "what's the simplest sufficient mechanism?", the team has predetermined the answer. The reframe: every AI initiative starts with "does this need to be AI?" If the answer is no, ship rules. If yes, the next question is "does this need to be more than an API call?" Each rung must be earned by evidence.

Trap 2: Fine-tuning for prestige. Fine-tuning sounds impressive in the deck. It's defensible to leadership. It signals technical sophistication. It also commits the team to 6+ months of work, $40K+/month deployment cost, and ongoing maintenance — all to outperform RAG by an unmeasured amount on metrics that often don't matter. The discipline: require an evidence-grade A/B comparison against RAG before approving any fine-tuning budget. Most fine-tuning proposals don't survive the requirement.

Trap 3: Building commodity. The off-the-shelf API surface keeps expanding. Building capabilities that the API will ship better in six months is a write-off in advance. The discipline: build bespoke only where the bespoke layer is durable — domain knowledge, proprietary workflow, integration depth, harness mastery. Build commodity, and the next API update obsoletes the work. The 4-month bespoke assistant from the third story is the public version of this trap.

Remember this

In practice

Step 1: Audit your AI portfolio against the four rungs. For each initiative, name the rung it's operating at and the rung it should be operating at. Mismatches are the budget reset opportunities.

Step 2: Require evidence-grade ROI proof for any rung-up. No fine-tuning budget without an A/B against RAG on production traffic. No RAG infrastructure without an A/B against API+prompt on the same traffic. The friction is the feature.

Step 3: Run a Boring AI experiment on every new initiative. Before approving the full budget, run a 1-week prototype at the lowest plausible rung. The financial-services rules-engine story is the pattern: 1 week, $400/month, ships in 14 days. If the lowest rung is sufficient, ship it. If it's not, you've earned the right to step up — with evidence.

Step 4: Treat off-the-shelf API surface as expanding, not fixed. When evaluating bespoke vs buy, assume the buy surface will expand in 6 months. Build bespoke only where the bespoke layer is durable — domain, proprietary workflow, integration depth, harness.

Step 5: Translate the rung decision into business language. Leadership doesn't care about RAG vs fine-tuning. They care about cost, time-to-ship, and durability. The translation: "Rung 2 (API) ships in 4 weeks at $X/month. Rung 3 (RAG) ships in 12 weeks at $Y/month and unlocks Z% accuracy improvement on metric M. Rung 4 (fine-tuning) ships in 6 months at $W/month and unlocks an additional V% on metric M, but locks us into a model architecture that may be obsolete in 18 months. Recommendation: Rung 3, with explicit re-evaluation of Rung 4 in 6 months."

The practice — visualized

References