Tool Use & The Call Loop
Scenario
You give an LLM a 'get_weather' tool. It outputs a JSON request to use it. But nothing happens.
How does the model actually get the weather data back so it can answer the user?
Mental model
An agent is a worker with a toolbox who keeps asking 'am I done yet?'
Giving an LLM tools doesn't automatically make it do things. The model just generates a JSON request to use a tool. Your code must intercept that request, run the actual tool, and feed the result back into the model in a continuous loop until the model decides the job is finished.
Deep dive
When you give a model tools, it doesn't execute them itself. It acts like a manager delegating tasks. The model outputs a structured request saying 'run the weather tool for Tokyo'. Your application code must catch this request, halt the model, and actually run the Python or API call.
So far: The model requests a tool; your app executes it.
Once your app runs the tool and gets the weather data, the model is waiting blindly. You must append the tool's result (the 'observation') to the conversation history and call the model again. This hands the baton back to the manager to decide what to do next.
So far: Feed the real-world result back to the model as a new message.
This creates a while-loop. The model asks for a tool, you run it, you give back the result, the model asks for another tool, you run it... The loop only breaks when the model decides it has enough information and finally outputs a plain text message to the user.
Tool use is a while-loop bridging the model's brain and your app's hands.
Code examples
The manual tool-call loop
messages = [{'role': 'user', 'content': 'What is the weather in Tokyo?'}]
while True:
# 1. Ask the model
response = client.chat.completions.create(model='gpt-4o', messages=messages, tools=my_tools)
msg = response.choices[0].message
messages.append(msg)
# 2. Break if model is done (no tool calls)
if not msg.tool_calls:
print(msg.content)
break
# 3. Otherwise, execute each requested tool
for call in msg.tool_calls:
if call.function.name == 'get_weather':
args = json.loads(call.function.arguments)
result = get_weather(**args) # Your actual Python function
# 4. Append the result (observation) and loop again
messages.append({
'role': 'tool',
'tool_call_id': call.id,
'content': str(result)
})Notice the `while True`. The model only does one step at a time; your code drives the loop, maps the requested name to your real function, and provides the `tool` role message so the model can read the output on the next iteration.
Common mistakes
- Thinking the model executes the code.: The model lives on an isolated server and only outputs text (JSON). Your application is what actually connects to databases, runs scripts, or hits APIs.
- Forgetting to append the tool response to the history.: If you don't feed the tool's output back into `messages` and call the model again, the model will just hang or repeat its tool request. It needs the observation to continue.
Glossary
- agent
- An AI setup where the model is given tools and allowed to repeatedly think, act, and check results until it finishes a complex task.
- observation
- The real-world result you hand back to the AI after running the tool it asked for, so it can decide what to do next.
- while-loop
- The continuous back-and-forth cycle in your code where the AI asks for a tool, you run it, and you send back the results until the job is done.
Recall questions
- What happens immediately after the LLM decides to use a tool?
- How does the model know the result of a tool execution?
- What condition breaks the tool-call while loop?
Questions & answers
A junior engineer complains that their agent 'keeps asking to run the weather tool over and over without ever answering the user'. What is the most likely bug in their code?
They are likely failing to append the tool's result (or the assistant's tool-call message itself) to the conversation history before calling the model again. Without the tool's output in the context window, the model has amnesia, sees the same original user prompt, and requests the same tool again.
Approach: Explain the agent loop: the model needs the 'observation' (the tool role message) to know the action succeeded and move on to the final answer.
Write a basic agent loop in Python that handles an LLM's tool calls and returns the final text response.
A while-loop that calls the LLM, checks for `tool_calls`, executes them, appends both the assistant's request and the tool's response to the message array, and loops until `tool_calls` is empty.
Approach: Look for the core `while` loop, the correct appending of the `assistant` tool-call message, and the `tool` role message mapping the `tool_call_id`. Correct role management is the primary failure point.