Enforcing a Spend Ceiling on a Self-Directed Agent

Before you start

You need three things in place, or the steps below have nothing to attach to.

• One choke point for model calls. Every request the agent makes must pass through a single client wrapper you control. If calls are scattered across tool implementations, subagents and retry helpers, consolidate them first — a ceiling enforced in four of five places is a ceiling with a hole in it.
• A price table per token class, per model. Not one number per model: separate figures for uncached input, cached input, output and (where the model exposes it) reasoning tokens. Pull these from the provider’s current price list and date the file; you will be recomputing money from token counts and stale prices give you a confidently wrong ledger.
• A task identifier that travels with every call. The ceiling is scoped per task, so each request has to know which task it belongs to. If you already emit spans with the OpenTelemetry generative-AI semantic conventions, the trace or a baggage entry is the natural carrier; if not, thread an explicit task_id through the wrapper.

Steps

1. Enforce the ceiling where the calls are issued, not in a runbook.

   A conventional service has a cost profile a developer can reason about in advance: this endpoint makes two queries and one outbound call, so a request costs roughly that. An agent does not. It chooses how many calls to make, how large each one is and how many tools it invokes, and it makes those choices at runtime in response to what it has seen so far. Two runs of the same task can differ by an order of magnitude in calls. Martin Fowler’s collection of memos on building software with generative models keeps returning to this non-determinism: the same prompt does not reliably produce the same behaviour, so nothing about the workload can be assumed from the code.

   The consequence is that “we agreed the agent should stay under a dollar a task” is not a budget. It is a hope, and the agent has never read it. The only place a budget can be real is inside the wrapper from the prerequisite: before each request goes out, the wrapper checks the running total for that task against the ceiling and refuses, downgrades or defers the call if it would breach. Anything enforced after the fact — a dashboard alert, a weekly report — records the overspend; it does not prevent it.

2. Scope the ceiling to the task, and only then roll it up.

   Most teams start with the number their finance system already has: a monthly cap on the provider account. Keep it as a backstop, but understand what it is. A monthly figure tells you the size of the damage after it happened. By the time it trips, the tasks that caused it have already run, and you have a bill and no idea which of them was responsible. A per-task ceiling stops the runaway task while it is running, and it isolates the blast radius: one task looping on a tool error burns its own allowance and nothing else.

   Set the per-task figure from data, not instinct. Run the task class a few dozen times with a generous provisional cap, take the distribution of realised cost, and put the ceiling at a point that lets normal runs finish but catches the tail. If you cannot see per-task cost yet, the page on attributing agent cost and latency to the work that caused it covers the tagging you need to get that distribution in the first place. Different task classes get different ceilings — a triage step and a multi-file refactor are not the same size of job — and the monthly cap becomes a sanity check on the sum, not the control.

3. Meter in money, not tokens.

   The number the provider returns on every response is a token count, and it is tempting to enforce the ceiling directly on it. Do not. Counting tokens is not counting money, because token classes price differently — cached input, uncached input, output and reasoning tokens each carry their own rate — and the mix varies with the shape of the task. A retrieval-heavy task with a large, stable prefix is mostly cached input; a task that writes long files is output-heavy; a task on a reasoning-tuned model may spend most of its budget on tokens the reader never sees. Two tasks with identical total token counts can differ in cost by several multiples depending on that ratio, so a token ceiling either starves the cheap task shape or lets the expensive one through.

   The wrapper therefore does the conversion on every response: multiply each class’s count by that class’s price for the model that was actually called, and add the result to the task’s ledger in currency. The OpenTelemetry generative-AI semantic conventions give you standard attribute names for the input and output token counts on the span, which is worth adopting so the ledger and the traces agree — but the spec records usage, not price. Money is a derived quantity you own, and the price table from the prerequisite is what derives it. When a task is routed across more than one model, the per-model prices matter even more; see routing requests between models for how that routing decision interacts with cost.

4. Decide, in code, what happens on breach.

   A ceiling with no defined breach behaviour has one anyway: it stops, and it stops at the moment the task is most expensive to abandon — deep into a run, holding partial work, with the reader waiting. Choosing silently is choosing that. Pick one of three behaviours explicitly, per task class, and put it next to the ceiling in configuration:

   • Stop. Halt the task, persist whatever state exists, and surface the breach to a human with the ledger attached. Right for tasks where a partial result is worse than none, or where the cause is almost always a fault (a tool loop, a malformed prompt) that money will not fix.
   • Degrade. Continue on a cheaper model, or with retrieval and tool use trimmed, and mark the output as produced under a degraded budget. Right for tasks where a slightly worse answer delivered beats a good answer never delivered — but only if the cheaper model can actually do the job. The trade-offs are the subject of frontier versus mid-tier model selection per task class; do not degrade to a model you have not tested on that class.
   • Queue. Pause the task and resume it when a human raises the ceiling or when a fresh budget window opens. Right for batch work with no one waiting on the result, and wrong for anything interactive.

   Whichever you choose, log the breach as an event with the task identifier, the ledger total and the behaviour taken. A ceiling that trips silently is indistinguishable from a bug.

5. Charge retries to the same ledger, and stop them restarting from zero.

   Retries are the line item that surprises, and they surprise because they are usually implemented one layer below the budget. A task that fails at 90 % and restarts from the beginning bills the first 90 % again; if the retry helper does not draw from the task’s ledger, the ceiling never sees the second attempt, and a task nominally capped at one unit of spend can cost two, three or however many attempts the helper allows. The AWS Builders’ Library piece on timeouts, retries and backoff with jitter makes the general case — retries multiply load on a system that is already struggling, and the remedy is a bounded retry budget with backoff and jitter rather than unlimited immediate re-attempts. For an agent the same logic applies to money as to load. Concretely:

   • Route every retry through the same wrapper, so the retried call debits the same task ledger as the original. There is no second budget.
   • Bound retries per task, not per call, and back them off with jitter so a provider-side incident does not turn every in-flight task into a synchronised storm — the interaction with provider limits is covered under agent execution under rate limits and concurrency caps.
   • Checkpoint completed steps so a retry resumes from the last good state rather than the start. The 90 % that was paid for should not need paying for twice; if the run’s accumulated context is what makes it expensive to resume, that is a context-management problem as much as a retry one.
   • Distinguish retryable failures (transient provider errors, rate limits) from non-retryable ones (validation errors, a tool that will fail the same way again). Retrying the second kind is pure spend.

Expected result

Every model call the agent makes passes through one wrapper that knows the task it belongs to, converts the response’s token counts into currency using per-class, per-model prices, and adds the result to a per-task ledger. Before each call the wrapper compares that ledger against a ceiling set from the observed cost distribution of the task class. On breach it takes a behaviour you chose in advance — stop, degrade or queue — and emits an event carrying the task identifier and the total. Retries debit the same ledger, are bounded per task, back off with jitter, and resume from a checkpoint rather than from the start. The monthly account cap still exists, but it trips only if the per-task ceilings have been set wrong, and when it does the ledgers tell you which task class to fix.