Before I write another system prompt that tells the model it is a "careful senior engineer," I want a budget object. Personality is optional. Limits are not.
Most AI features I review start with tone, tools, and a long instruction block. The budget shows up later as a magic number in an env var, or not at all. Then someone demos a happy path, the feature ships, and a single stuck loop burns through the monthly spend while a user stares at a typing indicator that never resolves.
I am not against good prompts. I am against shipping a feature whose only stop condition is hope.
What I mean by a budget object
A budget is the explicit envelope the run is allowed to spend before the product must fail closed or ask a human. Not vibes. Fields.
1type AiFeatureBudget = {2 maxInputTokens: number3 maxOutputTokens: number4 maxSteps: number5 maxToolCalls: number6 maxWallTimeMs: number7 maxCostUsd: number8 onExhausted: "fail" | "ask_user" | "degrade"9}
That type is deliberately small. You can add per-tool caps later (`maxShellCalls`, `maxWebFetches`). You can attach different budgets to preview vs production. The point is that the envelope exists as data the runtime enforces, not as a sentence the model might ignore.
I put this next to the feature config, beside the model id and the tool allowlist. If a PR adds an AI path without a budget, review should bounce it the same way it bounces a queue consumer without a timeout.
Failure modes when the budget is missing
**Runaway loops.** The model calls a tool, sees a soft failure, retries with a slightly different argument, fails again, and keeps going. Without `maxSteps` and `maxToolCalls`, you have invented a while(true) that invoices you. I have seen agents re-read the same failing schema validation twenty times because nobody capped steps and the "be persistent" line in the prompt won.
**Surprise invoices.** Token caps on the provider help, but they are not a product budget. A feature that fans out into embeddings, a judge model, and three tool rounds can stay under a single completion limit and still blow a cost ceiling. `maxCostUsd` forces you to estimate unit economics before launch: what is one successful run allowed to cost, and what happens on a pathological one.
**Stuck UX.** Wall time is the limit users feel. If your API route waits on an agent with no `maxWallTimeMs`, the browser times out first. Then the user retries. Now you have two runs, no user-visible progress, and a support ticket that says "it spun forever." Fail closed with a clear error beats a spinner that outlives the HTTP connection.
**Silent degradation.** Sometimes teams "fix" runaway cost by quietly lowering model quality mid-run with no signal. That is still a budget policy — just an accidental one. Put `onExhausted` in the object so product can choose: hard fail, ask the user to continue with a new budget, or degrade to a cheaper path with an explicit banner.
Budgets before personality
Personality prompts are cheap to write and expensive to trust. "You are concise" does not stop a 40-step tool chain. "Never spend more than necessary" is not a cost ceiling. The model is not your finance system.
Order of operations that has worked for me:
1. Define the user-visible job (draft a PR summary, triage a webhook, answer from docs). 2. Write the budget for a normal run and a worst-case run. 3. Choose model and tools that fit inside that envelope. 4. Then write the prompt and tone.
If the job cannot fit the budget, the job is wrong or the budget is wrong. Do not paper over that with a longer prompt.
Enforce it in the runner, not in the prose
Check the budget at the same points you already check auth:
- before starting a step
- before each tool call
- after each model response (token and cost accounting)
- on a wall-clock timer independent of the model stream
When a limit trips, persist why. `exhausted: maxToolCalls` in the run record is useful. A truncated chat that just stops is not. Ops needs to see whether users are hitting walls because the feature is under-budgeted or because the agent is looping on bad tool results.
Idempotent retries matter here too. If a request dies at 29 seconds against a 30-second wall budget and the client retries, the new run needs its own budget — do not inherit a half-spent ghost from a dead worker without knowing it.
Different surfaces, different envelopes
A docs chat on the marketing site should have a tight wall time and almost no tools. An internal coding agent in a sandbox can afford more steps, but still needs a cost ceiling and a tool-call cap so a bad eval case cannot open forty shells. A cron that classifies overnight tickets can use more wall time and still require `maxCostUsd` so a bad deploy does not become a weekend bill.
Same type. Different numbers. That is the whole pattern.
What I want in the PR
When someone adds an AI feature to the repo, I look for the budget object before I look at the prompt. Show me the defaults. Show me what happens on exhaustion. Show me how cost and steps are recorded on the run.
If those are missing, we are not arguing about copy. We are arguing about whether this feature is allowed to become an unbounded job with a friendly UI.
Give it a budget first. The personality can wait until the meter has a hard stop.