System / User / Assistant Roles

Scenario

You have two instructions: 'Always reply in French.' In test A, you put it in the system message. In test B, you put it in the user message. Before you test, predict: will the results be different?

Think about whether role placement matters — then read on to find out why the same text in different roles produces different behavior.

Mental model

Three voices in every conversation: the director (system), the human (user), and the actor (assistant). The director's rules run before the human ever speaks.

Chat models do not take a single text prompt — they take a structured list of messages, each tagged with a role. The role tells the model who is speaking and how much authority that voice carries. System sets the rules before any user arrives. User is the human's turn. Assistant is the model's reply — and you can pre-fill the start of a reply to steer its format.

Deep dive

The system message is the developer's invisible script. It is set by the application, not the user, and it runs first — before any user message. This is where you set persona ('You are a concise legal assistant'), define rules ('Never reveal pricing'), and frame the task. The model is trained to follow system instructions with high reliability.

System = developer-controlled rules that run before any user speaks.

The user message is the human's turn. In a real chatbot, this is what the user typed. In an automated pipeline, it may be a string you build in code — a question, a document to summarize, a list of items to classify. A conversation may have several alternating user and assistant turns. This list is the 'history' that gets resent on every call.

User = input to process. Can come from a human or be built by your code.

The assistant message is the model's output. But you can ALSO include an assistant message in your list before the model replies. This is called a prefill — you give the model the start of its own answer, and it continues from there. Prefills are useful for forcing a specific output format, such as starting a JSON block.

Assistant = model output. Pre-filling it steers format or continuation.

Why does role placement matter? Models are trained on conversation data where the structure is always: system → user → assistant → user → assistant... The training teaches the model to treat system as the highest-authority voice and user as lower-authority input. Prompt injection attacks try to blur this boundary — more on that in the injection lesson.

The role hierarchy (system > user) comes from training, not a hardcoded rule.

**The same instruction in different roles gets different results.** 'Always reply in French' in the system message is followed far more reliably than the same text in a user message. The model has learned that system means 'the developer requires this', while user means 'the human prefers this'. Role placement is a form of prompt engineering.

System instructions carry more behavioral weight than the same words in a user message.

Code examples

The canonical messages[] structure

from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
    model='gpt-4o',
    messages=[
        {'role': 'system', 'content': 'You are a concise support agent for Acme Corp. '
                                      'Only answer questions about Acme products.'},
        {'role': 'user',   'content': 'How do I reset my Acme SmartLock password?'},
    ],
)
print(response.choices[0].message.content)

The system message is the application developer's lever — it sets rules the user never sees. The user message arrives fresh each call. Notice the system prompt encodes two rules: topic scope and persona.

Multi-turn conversation: re-sending history every call

history = [
    {'role': 'system', 'content': 'You are a helpful assistant.'},
]

def chat(user_message: str) -> str:
    history.append({'role': 'user', 'content': user_message})
    resp = client.chat.completions.create(model='gpt-4o', messages=history)
    reply = resp.choices[0].message.content
    history.append({'role': 'assistant', 'content': reply})
    return reply

print(chat('My order number is 4821.'))   # model learns the order number
print(chat('What was my order number?'))  # can recall it — it is in history

There is no server-side memory. Every call resends the full `history[]` list. The model 'remembers' previous turns only because you re-include them. Remove old turns when the list approaches the token limit.

Assistant prefill to force output format

response = client.chat.completions.create(
    model='gpt-4o',
    messages=[
        {'role': 'system',    'content': 'Extract key facts from the text.'},
        {'role': 'user',      'content': 'The meeting is on Tuesday at 3pm in Room B.'},
        {'role': 'assistant', 'content': '```json\n'},  # prefill: force JSON block start
    ],
)
# Model continues from the prefill: '{"day": "Tuesday", ...}'

Prefilling the assistant message forces the model to continue from where you started. Starting with `'```json\n'` almost always produces a valid JSON block. Note: this technique works more reliably with the Anthropic Claude API than with OpenAI.

Common mistakes

Putting application rules in the user message: Developer-controlled rules belong in the system message, not the user message. System instructions have higher authority and are harder for users to override. A user message saying 'ignore your previous instructions' has less effect on a rule set in the system message.
Not resending history and wondering why the model forgot: The API is stateless. Every call starts fresh. If you want the model to remember turn 1 at turn 5, you must resend turns 1–4 in the `messages[]` list. This is the main reason chatbots seem to forget the user's name.

Glossary

system message: The developer's instruction that runs before any user message. It sets the AI's rules and persona, and carries the highest authority.
user message: The input from the human — a question, a command, or a document to process. The AI reads this and replies to it.
assistant message: The AI's reply in the conversation. You can also include a partial assistant message to steer how the AI continues its answer.

Recall questions

Explain the three message roles to a classmate, in plain words.
Why does the model remember a user's name from turn 1 when it is now turn 5?
What is assistant prefilling used for?
What happens if you put your developer rules in the user message instead of the system message?

Questions & answers

A user complains that your chatbot asks for their account ID in every single message, even after they provide it. What is missing in your API implementation?

The API is stateless — every call starts fresh with no memory of previous turns. The developer is not appending the user's previous messages and the model's previous replies to the `messages[]` array before the next call. To fix this, resend the full conversation history with every new request.

Approach: Identify the root cause as the stateless nature of the chat completions API. Show that memory is managed manually by appending to the history list.

You're building a pipeline where GPT-4o reads a user-uploaded PDF and answers questions about it. Where do you put the document content — system, user, or assistant — and why?

In the user role. The document is input to process — it changes with every request, so it belongs in the user message. The system message is for developer-controlled rules that apply to all requests (persona, constraints). Putting the document there would mean it could not change per user.

Approach: The system/user boundary tracks developer-controlled vs per-request content. Documents vary per request, so they go in the user message. Fixed rules go in the system message.

Continue learning

Next: Completion vs Chat Models

Next: System Prompts & Guardrails

Return to AI Engineering Roadmap