Multi-Step Planning

Mental model

Multi-step planning is breaking a massive boulder into manageable pebbles before swinging the hammer.

If you give an agent a massive goal ('write a web scraper, run it, and graph the data'), a basic ReAct loop will get lost halfway through. Multi-step planning forces the agent to write out a comprehensive plan (a checklist or dependency graph) upfront, and then check off sub-tasks one by one. It shifts the architecture from a reactionary loop to a structured execution engine.

Deep dive

In a basic loop, an agent only thinks about its immediate next action. For long horizons, it loses track of the overarching goal, gets stuck in error loops, or forgets what it was trying to do. It lacks a map.

So far: Pure reactive agents get lost on long, complex tasks.

A Planning Agent separates 'planning' from 'execution'. First, a Planner prompt takes the user's goal and generates a strict JSON array of ordered sub-tasks. This becomes the fixed itinerary.

So far: Step 1 is generating a strict checklist of sub-tasks.

Next, an Executor agent takes the first sub-task from the list and focuses entirely on solving it. Once completed, the result is appended to a shared memory, and the Executor moves to the second sub-task. If things go wrong, the Planner can be called again to replan.

Planning separates the 'manager' who writes the checklist from the 'worker' who executes each step.

Code examples

The Planner (generating the task list)

def generate_plan(user_goal):
    response = client.chat.completions.create(
        model='gpt-4o',
        response_format={'type': 'json_object'},
        messages=[
            {'role': 'system', 'content': 'You are a planner. Break the user goal into a JSON array of sub-tasks.'},
            {'role': 'user', 'content': user_goal}
        ]
    )
    # Returns: { "tasks": ["Find company API URL", "Fetch data", "Format to CSV"] }
    return json.loads(response.choices[0].message.content)["tasks"]

We use a separate, fast call to strictly generate the roadmap. Structured outputs (JSON mode) guarantee we get an array we can loop over in Python, rather than a conversational response.

The Executor (working the checklist)

tasks = generate_plan("Research the current CEO of Apple and email me their bio.")
context = []

for i, task in enumerate(tasks):
    print(f"Executing Task {i+1}: {task}")
    
    # The Executor only sees the current task and previous results
    prompt = f"Task: {task}\nContext from previous steps: {context}"
    
    result = execute_agent_loop(prompt, tools) # Standard ReAct loop
    
    # Save result for the next task to use
    context.append(f"Result of '{task}': {result}")

print("All tasks complete. Final summary:", context[-1])

This Python loop enforces the plan. The executor agent doesn't have to worry about the big picture; it is given a highly focused prompt ('Task 2: Fetch data') and the context of what just happened. This massively reduces hallucinations.

Common mistakes

Trying to do planning and execution in a single massive prompt.: Asking the model to 'plan the steps and also do the first step' confuses it and stretches its attention. Separation of concerns (one call to plan, separate calls to execute) is much more robust.
Making the plan totally rigid.: In the real world, step 1 might reveal that step 2 is impossible. Advanced architectures allow the Executor to fail and return an error to the Planner, which then generates a revised plan.

Glossary

Multi-step planning: A strategy where the AI writes down a complete checklist of sub-tasks before it starts working, keeping it from getting lost.
Planner: The part of the AI system that acts like a manager, taking a big goal and breaking it down into a strict itinerary of tasks.
Executor: The worker part of the AI system that takes one sub-task at a time from the checklist and focuses entirely on finishing it.

Recall questions

What is the primary advantage of a multi-step planning architecture over a basic ReAct loop?
In a Plan-and-Execute system, what does the Executor agent focus on?
Why is JSON mode or structured output particularly useful in the Planning phase?

Questions & answers

Describe the Plan-and-Solve (or Plan-and-Execute) pattern. When would you use it over a standard agent loop?

Plan-and-Execute splits the workflow into two phases: a Planner LLM creates a step-by-step checklist, and an Executor LLM performs each step sequentially. You use this for long-horizon tasks (like 'write a research report') where a standard ReAct loop would get distracted, forget the original goal, or get stuck in an endless loop of minor errors.

Approach: Highlight separation of concerns: separating the 'manager' who defines the DAG of tasks from the 'worker' who executes a highly-focused sub-task.

In a planning agent, what happens if Step 2 on the checklist fails completely?

A naive system will just crash or pass garbage to Step 3. A robust system catches the failure, halts the Executor loop, and passes the error back to the Planner LLM. The Planner is instructed to generate an updated, revised plan based on the failure.

Approach: Demonstrate knowledge of dynamic replanning. Static lists are brittle; true agents need a feedback loop back to the planner.

Return to AI Engineering Roadmap