Function / Tool Calling

Roadmaps › AI Engineering

Scenario

A user asks your chatbot: 'What is the weather in Delhi right now?' The model's training data is from 2024. Before you read on, predict: can you solve this by just including a weather URL in the system prompt?

Think about what the model can and cannot do — then read on to learn how tool calling enables real-world actions.

Mental model

The model does not run your tool — it writes a job ticket, and your code executes the work.

Tool calling is a structured round-trip. You tell the model which tools exist (name + schema). The model reads the user message and decides whether to call a tool. If it does, it returns a structured JSON job ticket instead of a text answer. Your code reads the ticket, runs the actual function, and sends the result back as a new message. The model then reads the result and generates its final text answer. The model is the planner; your code is the executor.

Deep dive

You register tools by passing a `tools[]` list. Each tool has a name, a description (critical — the model uses this to decide when to call it), and a parameter schema in JSON Schema format. The model reads this list alongside the user message. If it decides a tool is needed, instead of returning text in `message.content`, it returns a `tool_calls` list with the function name and arguments. `finish_reason` is `'tool_calls'`, not `'stop'`.

Model signals a tool call via finish_reason='tool_calls' and a structured JSON ticket — not by running anything.

Your code does the actual work. You read the tool name and arguments from the response, run the matching Python function, and get a result. You append two new messages to the history: the assistant's `tool_calls` message (from the model) and a `tool`-role message with the result string. The model sees the result on the next API call and generates the final text answer. The round-trip always takes at least two API calls.

Six-step round-trip: model asks → your code runs → result sent back → model answers.

The tool description field is critical — the model uses it to decide when to call the tool. A vague description ('gets data') leads to wrong trigger decisions. A precise description ('Returns current temperature and conditions for a given city. Call when the user asks about current weather.') lets the model call it exactly when needed. Treat the description as a trigger policy, not a comment for humans.

Tool description quality = call trigger accuracy. Write it for the model, not for a human reader.

The model can request multiple tools in one response — a `tool_calls` list with several entries. You run all of them, then send all results back in one follow-up request. This is used in agents to parallelize independent lookups — get weather + get calendar + search docs simultaneously — cutting the number of round-trips.

Parallel tool calls: model issues N tickets; run all N; send all results back at once.

Code examples

Define a tool and send the first request

from openai import OpenAI
import json
client = OpenAI()

def get_weather(city: str) -> dict:
    return {'city': city, 'temp_c': 38, 'condition': 'sunny'}  # real API call here

tools = [{
    'type': 'function',
    'function': {
        'name': 'get_weather',
        'description': 'Returns current temperature and conditions for a city. '
                       'Call when the user asks about current weather.',
        'parameters': {
            'type': 'object',
            'properties': {
                'city': {'type': 'string', 'description': 'City name, e.g. Delhi'},
            },
            'required': ['city'],
        },
    },
}]

messages = [{'role': 'user', 'content': "What's the weather in Delhi right now?"}]
response = client.chat.completions.create(model='gpt-4o', messages=messages, tools=tools)
print(response.choices[0].finish_reason)  # 'tool_calls'

The model returns `finish_reason='tool_calls'` with a JSON ticket specifying `get_weather(city='Delhi')`. Your code has not run anything yet — the model only planned the call.

Execute the tool and send the result back

assistant_msg = response.choices[0].message
tool_call = assistant_msg.tool_calls[0]

args = json.loads(tool_call.function.arguments)   # {'city': 'Delhi'}
result = get_weather(**args)                       # your code runs the real function

messages.append(assistant_msg)                     # the model's tool_calls message
messages.append({
    'role': 'tool',
    'tool_call_id': tool_call.id,
    'content': json.dumps(result),                 # result as JSON string
})

final = client.chat.completions.create(model='gpt-4o', messages=messages, tools=tools)
print(final.choices[0].message.content)
# 'It is currently 38°C and sunny in Delhi.'

The `tool` role message links the result back to the specific tool call via `tool_call_id`. On the second API call, `finish_reason='stop'` — the model has the result and generates a natural-language answer. This two-call pattern is the core of tool use.

Handle parallel tool calls

if response.choices[0].finish_reason == 'tool_calls':
    tool_messages = [response.choices[0].message]  # assistant message first

    for tc in response.choices[0].message.tool_calls:
        fn_name = tc.function.name
        args = json.loads(tc.function.arguments)

        if fn_name == 'get_weather':
            result = get_weather(**args)
        elif fn_name == 'get_calendar':
            result = get_calendar(**args)
        else:
            result = {'error': f'Unknown tool: {fn_name}'}

        tool_messages.append({
            'role': 'tool',
            'tool_call_id': tc.id,
            'content': json.dumps(result),
        })

    messages.extend(tool_messages)
    # Send one more API call with all results

When the model issues N tool calls in one response, run all N and send all results back in a single follow-up request. Each result must include its `tool_call_id` so the model knows which result matches which request.

Key points

Common mistakes

Glossary

job ticket
The structured JSON the model returns when it wants your code to run a tool — it contains the function name and the arguments to use.
JSON Schema
A structured format you use to describe each tool to the model — its name, what it does, and exactly what parameters it accepts.
parallel tool calling
When the model writes multiple job tickets in one response, letting your code run several tools at the same time before replying.

Recall questions

Questions & answers

Your tool-calling agent occasionally answers 'I called get_weather but I don't have the result yet.' What is the bug?

The tool result message is not being added to the message history before the follow-up API call. The model sees its own `tool_calls` message but no `tool`-role response, so it knows it asked for a result but has not received one. Fix: after running the function, append both the assistant's `tool_calls` message and the `tool`-role result message before making the second API call.

Approach: Trace the exact messages[] list: user → assistant(tool_calls) → tool(result) → assistant(final). A missing tool message is the most common tool-calling bug.

You have a tool called search_docs that the model invokes even for greetings like 'Hi there'. How do you fix this without removing the tool?

Improve the tool description. A vague description causes the model to call it speculatively. Change it to: 'Call ONLY when the user asks a specific question that requires looking up information in the company knowledge base. Do NOT call for greetings, small talk, or questions you can answer from general knowledge.' The description is the model's trigger policy.

Approach: Blame the description, not the model. The fix is a precise, policy-like description that tells the model exactly when to call and when not to.

Continue learning

Previous: Structured Output (JSON Mode)

Next: Tool Use & The Call Loop

Return to AI Engineering Roadmap