Structure as Control

The Story

A healthcare-tech team lifted physician quality scores from 3.1 to 4.6 out of 5 — same model, same prompt, same knowledge — by changing one thing: the output structure. The AI generates clinical summary notes after patient consultations: it listens to the conversation, processes the transcript, and produces a note the physician reviews and signs before it lands in the patient record.

Version 1 ships with a simple instruction: "Generate a clinical summary of this patient consultation." The model produces beautifully written prose summaries — fluent, comprehensive, well organised. The physicians hate them.

Not because the content is wrong. Because the format is wrong.

Physicians need specific sections in a specific order: Chief Complaint, History of Present Illness, Assessment, Plan. They need medication dosages in a standardised format. They need ICD-10 codes (the standardised classification system for diagnoses used in medical billing — acute appendicitis, K35.80) flagged for billing. They need allergies in a separate, prominently displayed section because missing an allergy is a patient-safety risk that lands in court.

The prose summary has all the information — buried in paragraphs. Finding the medication dosage requires reading the whole note. The ICD-10 code is mentioned in a sentence rather than tagged in a machine-readable field. The allergy information is in the second paragraph, not at the top of the note where physicians look first.

Version 2 adds a structured template — a JSON schema (a formal definition of what fields the response must contain and what format each field must follow) that requires specific sections with specific data types. Quality scores from physicians jump from 3.1/5 to 4.6/5. Same model. Same knowledge. Same prompt. Just a different output structure.

The constraint did not limit the AI. It focused it.

That is the lesson in one sentence: structure is not the enemy of quality. Structure is the quality.

The Core Idea

The Template layer of the seven-letter framework from The CONTEXT Stack does two jobs at once. It structures what goes INTO the context window — how all the other layers are organised — and it constrains what comes OUT — the shape of the model's response. Both jobs improve quality. They do it through different mechanisms.

Structure going in is about how the model processes information. From The Context Window you already know that position matters — beginning and end get strong attention, the middle dissolves. Template provides the organisational structure that makes navigation possible: clear section headers, labelled boundaries between layers, explicit markers for what is the system prompt versus what is retrieved knowledge versus what is conversation history. Without template structure, the model receives a wall of undifferentiated text. With it, the model receives organised, labelled information it can navigate.

Practitioners and research converge on a specific finding: 20-40% quality improvements from better templating alone, with no changes to the actual content. Same knowledge, same instructions, same tools — just better organised with clear XML tags, section headers, and labelled boundaries. The model processes the same information far more effectively when it is structured.

Structure going out is about reliability. When you constrain the output format — through JSON schemas, XML templates, or structured prompting patterns — you reduce the model's degrees of freedom. Instead of generating free-form text where it might produce a beautiful but unparseable narrative, the model fills in specific fields in a specific structure. A JSON response with a recommendation field and a confidence score can be processed by a UI component, logged by an analytics pipeline, and evaluated by an automated quality check. A prose paragraph that contains the same information requires parsing, extraction, and interpretation — three points of failure stacked on top of each other.

Think of it like: a form versus a blank page. Ask someone to describe their symptoms on a blank page, and you'll get a narrative that might or might not mention their allergies, their medications, the timeline. Give them a form with labelled fields — Allergies, Current Medications, When Symptoms Started, Pain Level (1-10) — and you'll get complete, structured, actionable information. The form does not limit what they can say. It ensures they don't skip the critical parts.

Four formatting approaches, each suited to different situations.

XML tags provide explicit hierarchy with labelled sections. They are verbose (more tokens) but unambiguous — the model never confuses where one section ends and another begins. XML is the preferred format for organising context input (structuring the seven CONTEXT layers). Anthropic's own guidance uses XML extensively for separating instructions, examples, and user content.

JSON schemas provide strict structure with type enforcement. They are ideal for output formatting when downstream systems need machine-readable responses. JSON schemas define required fields, data types, and allowed values — the model has to produce output that validates against the schema or the response gets rejected. (JSON — JavaScript Object Notation. The most common way computer programs structure data so other programs can read it cleanly.)

Markdown provides human-readable structure with minimal token overhead. Ideal for outputs humans consume directly — reports, summaries, analyses. Lower token cost than XML or JSON, but less reliably parseable by automated systems. Natural language templates provide the lightest structure — "Respond in three sections: Summary, Analysis, and Recommendation." Lowest token cost, easiest to write, but also the least enforced. The model may drift from the template over long conversations.

The PM's decision framework boils down to one question: what has to happen to this output after the model generates it?

• A human reads it directly → Markdown or natural language. Optimise for readability.
• Software parses it → JSON schema. Optimise for machine consumption.
• Both humans and software → JSON with a human-readable narrative field.
• It feeds another model → XML or JSON. Structure helps the next model the same way it helped this one.

The match between structure and task is critical. Over-constraining creative tasks strips the value out of using AI. Under-constraining data tasks creates parsing nightmares downstream. The PM decides where the product needs creativity (prose, analysis, brainstorming) and where it needs predictability (data extraction, classification, structured reports). The match between consumer and format is where structure earns its keep.

Where This Hits in Production

Structure makes evaluation possible. Machine Rubrics shows that structured outputs are dramatically easier to evaluate programmatically. A JSON response with a recommendation field can be checked against a rubric — an automated pass/fail. A prose response that buries the recommendation in paragraph three requires an LLM judge (an AI system that evaluates another AI's output) just to extract the recommendation before it can be evaluated. Structure removes the extraction step. That step is where most eval pipelines silently break.

Structured output increases per-step reliability in agent chains. In an agent pipeline where one model's output feeds another model's input, structure at each handoff point is what prevents cascade failure. (Cascade failure — when a small error in step 1 grows into a catastrophic error by step 7 because each step trusts the step before it. The reliability problem this multiplies into is unpacked in Agent Reliability.) If Step 1 produces structured JSON and Step 2 expects structured JSON, the interface is clean and testable. If Step 1 produces free text and Step 2 has to parse it, every parsing failure becomes a pipeline failure — and that failure rate compounds across steps.

Regulatory reporting. In regulated industries, AI-generated content often feeds into compliance reports, audit trails, and regulatory filings. These downstream systems expect specific formats — HL7 for healthcare data exchange, XBRL for financial reporting, specific XML schemas for government filings. The Template layer does not just improve quality. It makes the AI output compatible with regulatory infrastructure. A clinical summary that does not produce a parseable ICD-10 field cannot enter the billing system. The format choice is no longer a preference. It is a requirement.

Connecting the Dots

Template is the CONTEXT layer that ties all the others together. Constitution defines the rules. kNowledge provides the facts. Tracks provides the memory. Equipment provides the tools. eXecution returns the tool results. Template provides the structure that organises all of these into a coherent context window AND constrains the output into a usable format.

Think of it as the difference between a box of ingredients and a recipe. Every other CONTEXT layer is an ingredient. Template is the recipe — the structure that decides how the ingredients combine into something useful. Without it, you have good ingredients producing unpredictable results.

The harness connection. The harness — the system that wraps the model — is responsible for applying templates: loading the right template for the right query type, enforcing output schema validation, handling cases where the model's output does not conform. Template design happens at Level 1 (the PM defines the structure). Template enforcement happens at Level 2 (the harness implements it). The deep dive on the harness layer lives in What Is a Harness.

On the autonomy spectrum. Structure becomes more important as autonomy increases, not less. At Levels 1-3, a human reads the output and can mentally restructure a poorly formatted response. At Levels 5-7, the output feeds directly into automated systems — other models, APIs, databases, dashboards — with no human interpretation layer in between. Structured output is what makes agent-to-agent and agent-to-system communication reliable. The full autonomy framing lives in Spectrum of Agency.

The eval connection. Structure does not just enable evals. It defines what evals are cheap to run. A team that ships everything as JSON can build hundreds of automated test cases in a week. A team that ships everything as prose has to either build extraction logic for every test or pay an LLM judge for every evaluation. The total cost of the eval system is downstream of the format choice the PM made months earlier. For the deeper version of how the eval discipline shapes quality at scale, see The Eval Flywheel.

Remember This

1. Structure improves quality by 20-40% with zero content changes. Organising context input with XML tags and constraining output with schemas focuses the model's attention and ensures completeness. Same knowledge, same model, better answers.

2. Match structure to task. JSON for data tasks (machine consumers). Markdown for analysis (human consumers). Free text for creative tasks. Over-constraining creative tasks strips value. Under-constraining data tasks creates parsing failures. The match between consumer and format is the work.

3. Structured output enables automated evaluation, increases per-step reliability in agent chains, and makes AI output compatible with downstream systems. Structure is how you make AI outputs useful. Not just correct.

In Practice — The Template Library

Consider a B2B customer-success platform with an AI that generates account-health reports, meeting summaries, and renewal-risk assessments — three distinct use cases requiring three distinct templates. (A CSM — Customer Success Manager — is the person inside a software company who owns the relationship with each enterprise customer after the deal closes. Revenue Operations — Rev Ops — is the team that runs the data and forecasting for the entire revenue motion.)

The Account Health Report is JSON with narrative fields (consumer: CSM dashboard parses + human reads). The Meeting Summary is Markdown with required sections (consumer: human CSM reads in Notion/Slack). The Renewal Risk Assessment is structured JSON that feeds the risk model (consumer: Rev Ops dashboard + CSM). Renewal-risk satisfaction jumped +62% — from 2.9/5 to 4.7/5 — because the unstructured version was almost unusable for Rev Ops. The structured JSON feeds directly into the dashboard. Same content. Completely different utility.

That is the lesson of the Template layer in production: structure is not bureaucracy. Structure is the difference between an AI output that gets used and one that gets ignored. Three templates. Three different consumers. Three different formats. Each one earns its design choice. The PM is the architect of the match.