Production case study · Nick Cerutti
Production emergency: containing a $40K/day agent cost explosion
$40K+/day burn contained within 72 hours. Zero recurrence.
Late-stage platform, production multi-agent system. Incident response · priced after triage.
Context and scope
The $40K/day figure is a projected run-rate, not a claim that a full day of that spend was invoiced. This write-up describes one production incident and the controls shipped during the response.
A multi-agent pipeline in production began spending at a rate projected to exceed $40K per day — a $1.2M/month run-rate if left uncorrected — with no per-session budget controls and no circuit breaking at the tool-call layer.
What shipped
- Led triage and spend attribution first: per-agent, per-model, and per-tool cost mapping before touching the live system.
- Shipped circuit breakers at the tool-call layer with per-agent and per-tool thresholds.
- Enforced per-session spend caps, keyed to task type, at runtime.
- Deployed retrospective governance: deterministic policy validation on every tool call, so spend is blocked before execution rather than reviewed after the bill lands.
Reported outcomes
- Projected $40K+/day burn contained within 72 hours of engagement start.
- Zero recurrence since.
- The same controls shipped into the client's standard deployment path.
Anonymized client work, reported by Nick Cerutti. Client identities and raw operational records are not published.
What to check in your system
A cost dashboard can explain yesterday's bill. Runtime limits address a different problem: stopping an agent or tool call before it exceeds an agreed budget. The incident work combined attribution with enforcement, then put those controls on the normal deployment path.
- Can you attribute spend to the agent, model and tool involved?
- What stops a session from retrying indefinitely?
- Are those limits part of the standard deployment, or an incident-only patch?
Stack: AWS Bedrock · AgentCore · policy-as-code tool-call validation
Working on a similar problem?
Two weeks: an independent read of cost, reliability, and governance across your agent stack, with every finding tied to a dollar figure.
Agent Production Audit · from $12K