One corrupted null value broke a ledger agent's formatting tool. It tried to brute-force a fix: more than 15,000 reasoning calls in under an hour, about $50,000 of cloud spend, and database locks that halted live transactions. Nothing capped the calls, the cost or the failures, and every retry re-sent the whole history as billable input. Five controls.
The checks
Everything starts ticked. Untick what you will not do; the choice is saved on this device only, and Print keeps only what you kept.
What happened. The incident report's first control: cost-cap thresholds at the service-ID and project level. One key per agent, one budget per key; when it trips the agent stops, not the finance team.
The check.
- Give every agent its own API key or service identity; never a shared team key.
- Set a hard daily spend per identity at the provider or gateway, and a soft alert at half.
- A tripped budget stops the agent and pages a person; it never raises itself.
agent: ledger-reconcile identity: svc-ledger-agent # its own key, nobody else's daily_budget_usd: 50 # hard stop at the gateway; soft alert at 25
Source: threat-intelligence special report, AI risk and resilience in 2026, case study 6 (Sep 2026)
What happened. The loop was the agent retrying its own fix. The report asks for automated financial circuit breakers that halt an agent after a set number of consecutive task failures.
The check.
- Count failures per task, not per session; three in a row ends the run and files the task for a human.
- Treat a tool error, a schema failure and an empty result as failures; a retry is not progress.
- Log the last failing input with the task so the fix starts from evidence.
max_consecutive_failures: 3 on_trip: halt_task + notify_owner
Source: threat-intelligence special report, AI risk and resilience in 2026, case study 6 (Sep 2026)
What happened. Bounded recursion is the third control in the report. A task that needs more than the limit needs a person, and the limit is what turns a runaway loop into a ticket.
The check.
- Cap tool calls per task (25 is a sane default for a reconciliation step) and sub-task depth (2).
- Make hitting the limit a normal, logged outcome, never an exception that triggers a retry.
max_steps: 25 max_depth: 2
Source: threat-intelligence special report, AI risk and resilience in 2026, case study 6 (Sep 2026)
What happened. Tool output re-entering the context is billed again on every call. A September 2026 study measured a session's cumulative input at 14,293 times its first call; raw history adds 21 to 36 percent to its cost.
The check.
- Summarise tool output before it re-enters the context; keep the raw result in a store read by reference.
- Put a ceiling on context growth per task and on prompt size per call; measure input tokens per call, not only per session.
What happened. A loop shows up as calls per minute and input tokens per call climbing together, long before the invoice; the report's observability control would have caught this in minutes.
The check.
- Alert when calls per minute per identity and input tokens per call rise together for a few calls.
- Rate-limit each identity at the gateway so a loop degrades instead of spiking.
- Rehearse it: break a tool in staging and confirm the breaker, the budget and the alert fire.
Source: threat-intelligence special report, AI risk and resilience in 2026, case study 6 (Sep 2026)
The report gives the incident's figures ("over 15,000" calls, "~$50,000", "in under an hour") and names the controls (cost caps and bounded recursion at service-ID and project level, financial circuit breakers on consecutive failures, observability); it sets no threshold values. The numbers in the code blocks are starting points, not the report's.
Take this list with youThe PDF is the handout: this list on one or two pages. The Markdown is for your own docs, your wiki, or an assistant you want to hand the list to. Print gives you this list only, without the navigation.
Something in your production AI broke in a way that is not on this list? Tell me what broke.