Graceful Degradation

The Story

A healthcare documentation platform helps clinicians draft visit notes from voice transcripts. The product processes 85,000 transcripts per day across 400 hospital systems. On a Tuesday afternoon, the vector database hosting medical guidelines experiences a partial outage. Retrieval latency spikes from 120ms to 8,000ms, then starts returning timeout errors for 40% of requests.

Hospital systems on Integration A saw this: the pipeline stalled. Transcripts sat in the queue. Clinicians got a generic error: "Unable to process your request. Please try again later." Patient throughput dropped. The ED queue backed up.

Hospital systems on Integration B saw this: the pipeline detected the retrieval timeout within 200ms. It skipped guideline retrieval and generated the visit note using only the transcript, the clinician's prior note templates, and the model's parametric knowledge. A yellow banner appeared: "Note generated without current guideline references. Please verify clinical recommendations." The note was 85% as detailed. Patient throughput was unaffected.

Same outage. Same model. Same transcripts. Different degradation design. Integration A cost 2,400 delayed visit notes and measurable disruption to patient flow. Integration B: zero workflow interruption.

The Core Idea

Graceful degradation is the designed ability of the harness to maintain useful operation when individual components fail -- not by pretending failures didn't happen, but by explicitly reducing capability while preserving the core user outcome.

Think of it like a hospital operating during a power outage. Generators kick in within seconds -- not powering everything, but powering critical systems: operating rooms, ICU monitors, emergency lighting. Non-critical systems power down. The hospital continues to function at reduced capability, with staff aware of the limitations.

Circuit breakers per assembly stage. Each assembly stage has its own circuit breaker. The retrieval circuit breaker doesn't affect memory. A tool API failure doesn't take down knowledge retrieval. This stage-level isolation is what makes degradation graceful instead of cascading.

Degradation without communication is deception. If the system generated a response without retrieval, the user needs to know -- especially in high-stakes domains. Level 1 gets a subtle indicator. Level 2 gets an orange banner. Level 3 requires explicit disclaimers. The PM defines these messages. They're product copy that directly affects trust during the moments when trust is most fragile.

Where This Hits in Production

Rate limit degradation. When a production system hits model provider rate limits, it doesn't get an error -- it gets a 429. Without degradation design, users get errors during peak hours. With it, the system queues requests, routes overflow to a secondary model, or serves cached responses.

Cascading failure prevention. A slow retrieval service causes assembly to back up. Backed-up assembly exhausts connection pools. Exhausted pools cause memory service failures. Within minutes, a retrieval slowdown has taken down the entire product. Circuit breakers at each stage prevent this cascade.

In Practice: Same Outage, Different Design

The same ninety-minute outage produced opposite outcomes based entirely on whether degradation was designed in advance.

Remember This

1. The degradation ladder is a product specification, not an engineering implementation detail. The PM defines the levels, the minimum useful response at each level, and the user communication for each level.

2. Circuit breakers per assembly stage prevent cascading failures. A slow retrieval service should not take down the memory service, the tool router, or the template engine.

3. Degradation without communication is deception. If the system generated a response without retrieval, without memory, or with a simpler model, the user needs to know.