Preventing Agent Runaway: Budgets and Caps
Overview
An autonomous agent without limits is like a robotic vacuum with no off switch—if it gets confused, it will relentlessly bump into the same wall until the battery runs out. Step budgets and recursion limits act as the circuit breaker, stopping an agent from entering an infinite loop of failing tool calls.
Without structural limits, an agent that encounters a failing tool or logic gap can infinitely retry identical actions, rapidly consuming compute and costing thousands of dollars in API fees in mere minutes.
Where used: LangGraph state machines, ReAct orchestration loops, Production agent deployments
Why learn this
- Agents operate autonomously and can rack up massive API bills instantly if they enter an infinite ReAct loop.
- Controlling agent execution bounds is a non-negotiable core requirement for moving any LLM system into production.
- A hard step budget protects your system from infinite loops, whereas standard text generation limits do not.
Code walkthrough
from langgraph.graph import StateGraph
# Compile the graph into an executable app
app = graph.compile()
# Run the agent with a strict recursion limit to prevent infinite loops
final_state = app.invoke(
{'messages': [('user', 'Research quantum computing')]},
config={'recursion_limit': 15} # Stops execution after 15 node transitions
)Focus: The `recursion_limit` inside the `config` dictionary is the safety net. It explicitly halts the LangGraph execution if the agent transitions between nodes more than 15 times, definitively severing an infinite tool-calling loop.
Common mistakes
- Relying on max_tokens to stop agent loops: `max_tokens` limits a single inference generation. But an agent loop makes repeated, independent API calls. It can infinitely loop short, valid tool calls (like retrying a failing search) unless a hard step budget or recursion limit is enforced by the orchestrator.
- Assuming recursion_limit immediately cancels external side effects: A graph's `recursion_limit` only bounds the number of graph node transitions (supersteps). It does not cancel in-flight external tool calls, and it won't stop an infinite retry loop implemented entirely inside a single node's custom code.
Recall questions
- What is the functional difference between `max_tokens` and a step budget in an agentic architecture?
- Predict what happens if you set `max_tokens=200` but an agent repeatedly receives a '404' error from a web search tool and decides to retry it.
- Does setting a LangGraph `recursion_limit` stop an infinite while-loop located inside a specific node's logic?
Understanding checks
An agent tries to query a SQL database, but the schema has changed. The agent catches the error and decides to try the exact same query again. You have configured the LLM with `max_tokens=500` and the LangGraph orchestrator with `recursion_limit=10`. What happens?
The agent retries the failing query repeatedly until it hits 10 node transitions, at which point the orchestrator halts the execution and throws a recursion limit exception.
The `max_tokens` limit applies per-generation and won't stop the loop since each tool call generation is short. The `recursion_limit` correctly bounds the total number of cycles, acting as the ultimate circuit breaker.
Why can't you just add 'stop if you fail more than 3 times' to the system prompt to prevent runaway agents?
LLMs are probabilistic and may ignore instructions, especially when context windows grow large. System prompts provide soft guidance, whereas hard structural budgets implemented in the orchestration code guarantee termination.
Relying on the LLM to reliably count its own failures and halt itself is an anti-pattern. Orchestration safeguards must always be deterministic code.
Practice tasks
Implement a Hard Step Cap
Given a compiled LangGraph application `app`, invoke it with a user message to 'Analyze this data' and pass a configuration that ensures the graph will terminate if it exceeds 5 state transitions.
Continue learning
Previous: State machines for agents
Previous: Cost & Latency Budgeting