Begin With Consequence

First, some vocabulary

This series uses a few words precisely. If you are new to agent products, read this box once and the rest of the episode will move quickly.

Two more that get used constantly and are almost never defined:

Behavioral control — you tell the agent not to do something. A line in the prompt, a rule in the instructions. It works when the agent is behaving.

Structural control — you arrange the world so the agent cannot do it. The credential does not exist. The endpoint is unreachable. The gateway rejects anything above the limit. It works whether or not the agent is behaving.

Behavioral controls reduce how often something goes wrong. Structural controls bound what it costs when it does. This series is almost entirely about the second kind.

Who this episode is for, and what you will own

This is the first artifact in the series, and it is the one most likely to be produced by accident if you do not produce it on purpose. Every number in the Consequence Envelope will otherwise be set by whoever configured the prototype in week one — a developer optimizing for the demo working, which is a completely reasonable thing for them to have been optimizing for.

Engineering enforces the boundary. You decide where it sits. If you do not decide, the boundary stays wherever the prototype left it.

None of this requires knowing what a container is.

Where this sits in the series

The prologue argued that the environment is a product boundary, not infrastructure you inherit. This episode makes the first decision at that boundary, and the four artifacts that follow all reference it.

• Episode 01 — this oneBounds what one run may cause
• Episode 02Builds the world the run reasons inside
• Episode 03Establishes whose authority it carries
• Episode 04States what still holds when it is compromised
• Episode 05Proves all four are still true in production

The dependencies are real, not decorative:

• Episode 02's fallback behavior is expressed using the consequence classes defined here
• Episode 03's permissions are bound to the ceilings defined here
• Episode 04 exists partly because the per-run bounds defined here do not add up across many runs
• Episode 05 requires evidence that these bounds actually held

Get this artifact wrong and the four downstream artifacts inherit the error silently.

01Autonomy is not a dial

The industry talks about autonomy as a slider — less autonomous on the left, more on the right. It is an intuitive model and a false one. You can watch a major vendor decline to build it.

OpenAI's Codex ships two separate settings. Sandbox mode controls what the agent can technically touch. Approval policy controls when it must stop and ask a human. They are configured independently, three values each, which produces nine combinations rather than one slider position.12

Look at the two corners. Top-left is a system that can hardly do anything and interrupts you constantly. Bottom-right can reach anything and never pauses. Neither is more autonomous — they differ on two axes at the same time, and a single number cannot express that.

The sandbox governs reach. The approval governs consequence. Neither substitutes for the other.

Practical implication: if a vendor offers you one autonomy setting, they have made both decisions for you and not told you which way. That is a question for the sales call.

02Four dimensions of consequence

Consequence has four dimensions. They are independent — each can be tightly controlled while the others are wide open, and no single mechanism covers more than one.

• ReachWhich systems the run can touch at all
• AuthorityWhose power the operation carries
• PersistenceWhat remains after the run ends
• RecoverabilityWhether the effect can be undone — by whom, how fast, at what cost

They do not get equal attention, and the imbalance is predictable.

Reach gets discussed because it looks like architecture and architects are in the room. Recoverability is almost never discussed before launch, which is why it gets discovered by the support team at 11pm.

The trap: an agent that touches only one system, carries a narrow permission, and leaves nothing behind — but performs an action nobody can undo — is not a bounded agent. Three dimensions were designed. The fourth decided the outcome.

03Consequence is not the same as likelihood

This is the distinction most often collapsed by teams building their first agent, and collapsing it produces two opposite mistakes.

Consequence is how bad it is when the agent is wrong. Likelihood is how often the agent is wrong.

They are governed by completely different work. Likelihood is improved by better models, better prompts, better retrieval, better evaluation — the entire discipline covered in the AI Evals series. Consequence is bounded by the environment, which is this series.

A high accuracy rate does not shrink the blast radius of a single wrong action. A tight blast radius does not make a bad agent good.

How to hold both: the envelope is designed against the worst single outcome, assuming every component behaved exactly as specified. Evaluation is designed against the rate. You need both artifacts, and they are produced by different conversations with different people.

This distinction also settles a recurring argument. When someone says "the model would never do that," they are making a likelihood claim. The envelope is not interested in likelihood. It asks: if it did, what would it cost?

04Classify by reversibility, not by value

Before any dimension can be bounded, actions need classes. Three are enough.

Four rules make this useful rather than decorative.

Rule 1 — Sort by reversibility first, not by rupees. A ₹200 action that cannot be undone and a ₹200 action that can be undone in ten seconds are not the same risk. Value is the second sort key, not the first.

Rule 2 — A read can be C3. If the agent reads data the customer never agreed to share, or the read itself is logged and disclosed, that is real consequence. "It only reads" is not a safety argument.

Rule 3 — Classify actions, not agents. The refund agent operates at all three classes inside a single run. Calling it "a C3 agent" throws away the information you need to bound it.

Rule 4 — C2 is not free. A proposal a human rubber-stamps under time pressure is functionally C3 with an extra step and a false audit trail. Whether C2 is real depends on whether the reviewer sees authoritative data and has time to think — a question Episode 03 develops in detail.

An agent does not have a consequence class. Each of its actions does.

The class is a live control, not a label

This is worth understanding now, because it is where Episode 02 plugs in.

When the environment cannot verify that the agent's data is current, the run does not have to fail. It can drop from C3 to C2 — continue investigating, continue recommending, stop acting.5 The customer still gets value. Nothing irreversible happens on unverified data.

That mechanism only exists because the classes were defined here, at design time, with names the runtime can use.

05Reach — which systems it can touch

Reach is the dimension teams handle best, because it looks like infrastructure and infrastructure teams are already good at it. Two failures survive anyway.

Failure one: classifying by what it's called. An environment named staging that holds production-shaped credentials and can resolve internal hostnames has production reach. The label is not a control. This exact failure is what made the July 2026 evaluation-environment incident possible.

Failure two: reach nobody can explain. Every reachable system should trace to a step in the workflow. Access that was added for a reason nobody now remembers is the purest form of drift, and it is always found during an incident rather than during a review.

Record reach as three lists. Episode 02 turns these into actual network configuration.

For the refund agent, required is short: read the CRM, look up the payment, call a bounded refund endpoint, send transactional email, write to the audit log.

The denied list is the one that earns its keep, and it should name things a reviewer would ask about — the admin console, bulk export, public software registries, the general internet. Naming them proves the decision was made rather than overlooked.

OWASP's guidance on excessive agency lands in the same place from the security side: minimize the tools an agent can call, prefer narrow tools over open-ended ones like "fetch any URL," and enforce permissions in the systems being called rather than trusting the model to restrain itself.3

The strongest control is an absence

The refund agent holds no credential that permits changing a subscription. Not restricted. Not approval-gated. Not blocked by a rule. No such credential was ever created, and the endpoint appears on no list the agent can see.

Every capability you provision has to survive scrutiny forever. Every capability you decline to provision passes automatically, at zero cost, permanently.

This is the cheapest control in the entire series, and new teams consistently under-use it because removing a capability feels like removing a feature. It usually is not — it is removing a capability the feature never needed.

06Authority — whose power it carries

Authority gets a full episode of its own. Episode 01 needs one thing from it: for each C3 action, what is the maximum the agent may commit, and where is that maximum actually enforced?

Both columns describe the same intention. Only the right column holds when the model is confused, manipulated, or simply mistaken.

A ceiling that lives only in the prompt is a ceiling that applies on days when the prompt is being followed — which is precisely not the situation you are trying to bound.

What to write in the envelope: the number, and the system that enforces it. Episode 03 decides how the permission is issued. Episode 04 tests whether it survives an attack.

What to ask engineering: "Where is this enforced, and what happens when the service that checks it times out?" If a timeout means the request goes through, the ceiling only exists when the network is healthy.

07Persistence — what is left behind

Persistence is the dimension most often assumed rather than specified. "The container gets destroyed afterward" is a statement about the platform, not a specification for your product.

Four things can outlive a run, and they have different owners:

• Workspace contentsfiles the agent wrote. Usually destroyed. Sometimes not
• Session statesaved snapshots that let a long task resume after a failure
• External statesessions and caches left open in other systems the agent connected to. This is the one nobody checks
• Evidencethe audit record, which must survive, and must not be editable by the agent that produced it

Write two lists. The second is the one teams skip:

There is a tension here worth flagging early, because it reappears twice.

Saving a snapshot so a four-hour task can resume after a network blip is a reliability feature in Episode 02. It is a liability in Episode 04, because a snapshot taken after the agent was compromised faithfully restores the compromise. Episode 01's job is just to establish that persistence is a decision somebody makes, so that later episodes have something to constrain.

08Recoverability — how a wrong action gets undone

This dimension determines whether an incident is an inconvenience or an event, and it is the one least likely to have an owner before launch.

Reversibility is a spectrum, not a yes/no

Row four is where first-time teams misclassify. A refund that can be reversed in the accounting ledger has not been undone in any sense the customer experiences. They saw money arrive and then leave. That is two events, not zero.

The four-part test. For any action, name: the reversal procedure, the person who performs it, the time window in which it works, and what the customer sees. If any one of those is unknown, treat the action as irreversible and classify it accordingly. You can always relax this later with evidence.

Find the irreversible step

Every C3 workflow has one moment after which the world has changed. For the refund agent, it is the instant the payment system accepts the submission.

That single point governs behavior across three later episodes:

• Episode 02before it, restarting the run cleanly is safe. After it, restarting is prohibited and the run must reconcile against the payment system
• Episode 03a resumed run must not assume its permission went unused
• Episode 04recovery restarts from a verified checkpoint taken before the compromise; it never resumes into uncertain state

Identify the irreversible step during design, or identify it during an incident from the wrong side of it.

Doing it twice is the default failure

Here is a situation that will happen, and that reasoning cannot solve.

The agent submits a payment. The connection drops before a response arrives. From inside the run, three completely different situations look identical:

1. The request never arrived
2. The request arrived and failed
3. The request succeeded and the confirmation was lost

The agent cannot tell these apart. If it retries, situation three becomes a duplicate refund.

Idempotency — the word you will hear engineering use — means an operation can be repeated safely: sending the same request twice, carrying the same identifying token, has no additional effect if the first one worked. AWS describes it exactly this way in its guidance on making retries safe.4

The detail that matters, and that PMs are uniquely positioned to catch: the token must be derived from the business intent — this case, this transaction — not from the attempt. A retry that generates a fresh token is not a retry. It is a second refund with good manners.

Row one is a product decision wearing engineering clothes. It says: this business would rather a refund fail than happen twice. For money that is almost always right. For sending a status notification it might not be. Only you can say.

09Derive the ceiling; do not round it

Here is where most envelopes quietly fail. Someone proposes ₹2,000 because it sounds sensible. Nobody has a better number. A round figure becomes policy, and six months later nobody can explain it to an auditor.

A defensible ceiling comes from one of four places:

1. What humans actually decided. Pull the distribution of decisions the team doing this work made last quarter. A ceiling set at a real percentile of that distribution — say, the level below which 80% of their approvals fell — is defensible in a way an invented number never is.

2. An existing delegated authority. Most organizations already have a documented approval limit for this role. Inheriting it is legitimate, fast, and already survived a governance conversation.

3. The reversal window. If refunds below some threshold can be clawed back and above it cannot, that threshold is already doing real work. Use it.

4. Accepted aggregate exposure, divided down. Start from the monthly loss the business will accept and work backward through expected volume. This is the only method that connects the per-action number to the number that actually matters.

A ceiling nobody can source is a ceiling nobody will defend the first time it is tested.

The conversation to have: ask the domain owner for last quarter's decision distribution, segmented by category. If nobody can produce it, that is your first real finding — you are about to automate a process whose actual behavior has never been measured. That is worth knowing before you build, not after.

10What to do when there is no data

The advice above assumes a human process exists to measure. Often it does not. The product is new, the workflow is new, or the work was previously distributed across people who never logged decisions.

Do not let this become a reason to skip the number. Four workable substitutes, in descending order of strength:

1. Shadow mode first. Run the agent with no C3 capability at all. Let it produce recommendations that humans review and act on independently. After a few hundred cases you have the distribution you were missing — generated by your own agent, on your own population. This is the strongest option and costs one milestone.

2. Borrow the adjacent limit. Find the closest existing delegated authority in the organization — a different team, a different product, a manual process — and start there, explicitly marked as borrowed. A sourced-but-imperfect number survives review. An invented one does not.

3. Set the ceiling at the reversal boundary. If you know what can be undone, set the initial ceiling there. The worst case becomes "an action we can reverse," which is a defensible opening position for any product.

4. Accept a deliberately small number and schedule the revision. Pick a conservative figure, write down the evidence that would justify raising it, and set a date to look. This is weakest, but it is honest, and it is enormously better than a round number with no story.

What to write in the envelope either way: the number, the method, and the evidence that would change it. A ceiling with a stated derivation — even a weak one — is an auditable decision. A ceiling without one is a guess that has been formatted.

11The envelope can also be too tight

Every section so far has argued for bounding. Here is the other failure, and it is just as real and much less discussed.

An agent that refuses constantly looks excellent on every safety metric. No incidents. No unauthorized actions. Perfect control integrity. And it may be completely worthless, because every case it touches ends up back with a human, plus the latency of having tried.

Episode 05 names this directly: a system can refuse so much that control looks perfect and the product becomes useless.7

The costs of over-restriction are real, they just land on different teams than the costs of over-reach:

That last row is the dangerous one. A boundary people route around is worse than a wider boundary people respect, because the workaround has no envelope at all. If your refund agent can only handle ₹500 and the team's actual cases cluster at ₹1,800, you have not built a safe agent — you have built an agent nobody uses and a manual process nobody is watching.

How to hold both sides

Write the cost of refusal into the envelope alongside the cost of action. For each refusal condition, state what happens to the case and who picks it up. When the two columns sit next to each other, the conversation stops being "is this safe" and becomes "what is the right trade," which is the conversation you actually want with a domain owner.

Design the envelope to be as wide as the evidence supports and no wider. Both halves of that sentence are load-bearing.

12The run budget is also a safety control

Cost per run is usually treated as a finance question that arrives after launch. It belongs in this artifact for a different reason.

A bounded run budget limits what a malfunctioning or manipulated agent can consume before anyone notices. Episode 04 makes the point sharply: an agent that can neither steal data nor move money can still exhaust a rate limit or burn a budget until the surrounding service degrades for everyone else.

The time limit has a second effect that is easy to miss. Episode 02 shows that data is checked for freshness when the run starts and then used throughout — so the longer the run, the wider the gap between what the agent believes and what is currently true. A four-hour run acting on account balances read at minute one is making decisions on stale information by minute two hundred.

The run budget is a cost control. Under attack it becomes a containment control. Under long-running tasks it becomes a correctness control. Specify it once; it does three jobs.

13One run bounded is not the fleet bounded

The envelope bounds one run. Your actual exposure is a property of all runs happening at once, and the two are related by a multiplication most teams never perform.

A thousand runs executing simultaneously, each perfectly compliant with a ₹2,000 ceiling, represent ₹20 lakh — two million rupees — of live exposure. Every single run is inside policy. Nothing has malfunctioned.

This matters more than it first appears because agent failures are often correlated by construction. A bad prompt template, a bad policy update, or a compromised dependency affects every run of that version at the same moment. Runs that look independent share a failure mode.

Episode 04 builds the controls that bound the fleet.6 Episode 01's obligation is narrower, and skipping it breaks the chain: state the accepted aggregate exposure here, so a later episode has a number to enforce against.

If no one with the authority to accept risk has stated the monthly number, the product does not have a risk decision. It has an assumption that will be discovered during a board conversation.

14An envelope that cannot stop a run is a document

Episode 02 makes this argument about its own artifact, and it starts here. A specification that cannot block anything is a description of good intentions.

Rows four and six are the ones commonly written as warnings that should be written as refusals. An agent that acts while unable to record what it did has removed your ability to reconstruct the incident later.

15Translating this to your product

The refund agent is a useful example because money makes consequence legible. If you are building something else, here is the same structure mapped across four common agent types.

The generalizable questions, whatever you are building:

1. What is the single worst thing one run can do while every component works exactly as designed?
2. Which of its actions cannot be undone in a way the affected person experiences?
3. What does doing that thing twice cost, compared with not doing it at all?
4. Which capability could you simply not provision, and what would you lose?

If you can answer those four for your product, you can write the envelope. The refund numbers are illustration; the structure is the transferable part.

16The artifact

17How to run the review

Ninety minutes. The domain owner and someone who owns the money must be in the room. This happens before the PRD is signed, not after the prototype works.

Minutes 0–10 · Frame it. State the six questions from the opening. Say explicitly that the goal is not to decide how autonomous the agent should be.

Minutes 10–25 · Classify. Walk every action and assign C1, C2, or C3, sorting by reversibility first. Expect disagreement here; it is the productive part.

Minutes 25–40 · Find the irreversible step. For every C3 action, run the four-part test: reversal procedure, person, window, what the customer sees. Any action failing the test is irreversible until proven otherwise.

Minutes 40–55 · Set and source the ceiling. Ask for the distribution. If it does not exist, choose from the four fallbacks in section 10 and record which one.

Minutes 55–70 · Reach and absence. Walk the three lists. For every permitted entry, ask why. Then ask the better question: which capabilities can we decline to provision entirely?

Minutes 70–80 · Aggregate. Multiply the ceiling by forecast volume. Ask who accepts that number by name. Do not leave without a name.

Minutes 80–90 · Refusals and their cost. Walk Part H. For each refusal, ask who picks up the case and whether they have the capacity.

Leave with: a filled Part A, a sourced ceiling, a named irreversible step, a named risk accepter, and a list of what nobody could answer. The last one is the most valuable output of the meeting.

18Anti-patterns

• "We'll start conservative and loosen it later."Loosening requires evidence you have not defined. Episode 06 exists because this sentence usually has no mechanism behind it.
• "The ceiling is ₹2,000."Sourced from what? A round number is a placeholder in the costume of a decision.
• "It's 97% accurate, so it's safe to act."Accuracy is a rate. The envelope bounds a single occurrence. Different question, different artifact.
• "It's low risk, it only reads."Reads expose data. Exposure is consequence.
• "The refund is reversible."On paper, before settlement, by someone who works business hours — and the customer sees both events.
• "It's in the system prompt."Then it holds while the prompt is being followed, which is exactly the case you are not bounding.
• "Retries are handled."Bounded retries against an endpoint with no duplicate protection are bounded duplicate consequence.
• "Each run is capped at ₹2,000."And a thousand concurrent runs are capped at ₹20 lakh.
• "Legal will tell us the limit."Legal will tell you what is permitted. Only the business can say what it is willing to lose.
• "Nobody would ever do that."A likelihood claim in an argument about consequence.
• "We made it as restrictive as possible."Then measure the escalation queue, and check whether anyone has started routing around it.
• "We'll define this after the pilot."The pilot configuration becomes the production image. That is the inheritance gap running exactly on schedule.

19What product owns

Engineering enforces every bound in this artifact. Seven of them cannot be inferred from a prototype, and none require infrastructure expertise.

1. Consequence classes. What separates observe, propose, and act for this workflow, judged by reversibility first.
2. The ceiling and its derivation. The number, and the evidence behind it — including "we had none, so we did this instead."
3. The irreversible step. Where the world changes, and what becomes prohibited after it.
4. Duplicate tolerance. Whether a failed action or a doubled action is worse. For money, nearly always the first. For notifications, sometimes not.
5. Reversal economics. Who performs it, in what window, at what cost, and what the affected person experiences.
6. The cost of refusal. What happens to work the agent declines, and whether anyone has the capacity to absorb it.
7. Accepted aggregate exposure. The monthly number, and the named person who accepted it.

Engineering can enforce any bound you specify. None of these seven can be discovered by building.

20The AI PM page

Field artifact · Episode 01 · print this

Before your team sets a single limit

How to use this. The Consequence Envelope is what your team produces together. This page is what you bring to the room to produce it. If block 2 cannot be answered, the envelope you write will be invented numbers with a template around them.

1 · Decisions you own

2 · Questions you must get answered

1. What is the worst thing one run can do with every component working correctly? A bad answer describes a bug. A good answer describes a permitted action.
2. Which actions cannot be undone in a way the affected person experiences? Ledger-reversible is not customer-reversible.
3. What did humans doing this work actually decide last quarter? If nobody can produce it, your ceiling is invented — pick a fallback from section 10 and say so.
4. Where is the ceiling enforced, and what happens when that service times out? If a timeout lets the request through, there is no ceiling.
5. What is the irreversible step, and what becomes prohibited after it? Restart, retry, and resume all change meaning at that point.
6. What happens on the second attempt after a dropped connection? If nobody has answered this, budget for duplicate consequence.
7. At forecast volume, what monthly exposure does this imply, and who accepts it by name? A per-run bound with no aggregate is an unbounded product.
8. Which capabilities are we deliberately not provisioning? If that list is empty, nobody has considered absence as a control.
9. When the agent refuses, who picks up the case, and can they cope? An unusable agent is also a failed agent.

3 · Assumptions to test

• That the ceiling is enforced where value moves, not where the agent reasons.
• That low value per action implies low total exposure.
• That reversibility is a property of the action rather than of a time window.
• That the human baseline was ever measured.
• That excluded actions are genuinely unprovisioned, not just undocumented.
• That "the container is destroyed" covers sessions left open in other systems.
• That someone in operations knows they own reversal, at night.
• That the run budget is only about cost.
• That a high accuracy number reduces the blast radius of one wrong action.
• That the escalation queue can absorb every case the agent declines.

4 · Conversations to schedule

5 · Signals this work was skipped

• The ceiling is a round number and nobody can source it.
• Consequence classes were copied from a different product.
• Nobody can name a capability that was deliberately excluded.
• The envelope exists in a document but not in a gateway.
• The team still discusses autonomy as one dial.
• Retries are described as "handled" with no mention of duplicates.
• No monthly aggregate figure exists anywhere.
• Reversal has no named owner outside business hours.
• The irreversible step has never been written down.
• Accuracy numbers are being used to justify reach decisions.
• Nobody has asked what happens to the cases the agent refuses.

The test

If you can answer block 2, the envelope is a ninety-minute conversation.

If four or more are unanswered, the envelope already exists. It was written in week one by whoever configured the prototype, it is now load-bearing, and everyone downstream is about to inherit it.

21The line

Autonomy is not a dial. It is four independent decisions hiding behind one word.

A ceiling in the prompt is a preference. A ceiling in the credential is a bound.

Accuracy changes how often you are wrong. The envelope changes what being wrong costs. You need both, and they are different conversations.

Identify the irreversible step during design, or identify it during an incident from the wrong side of it.

You now know how much consequence one run may create. That limit is a statement about the world the run is placed in — and that world does not build itself. Episode 02 turns the reach grade into something reproducible: what must be present, what must be absent, and what must be true before the agent starts.