Completion vs Chat Models

Roadmaps › AI Engineering

Mental model

A completion model is a text autocomplete engine. A chat model is the same engine, retrained to play the role of a helpful conversation partner.

Both model types do the same thing at their core: predict the most likely next token given all the text before it. The difference is what they were trained to continue. A base completion model continues any text as if it were the middle of a document. A chat model was fine-tuned using RLHF on conversation data, so it learned to continue the 'assistant' turn of a structured dialogue — it knows what a good helpful reply looks like.

Deep dive

Base completion models like the original GPT-3 took a single string and continued it. If you typed 'What is the capital of France?', the model would output '...Paris, which is also known as the City of Light...' — a document continuation, not a clean answer. To get a direct answer, you had to prompt carefully: 'Q: What is the capital of France? A:'. The model then continued with 'Paris'.

Completion models: raw text in, text continuation out. No conversation structure.

Chat models (`gpt-4o`, `claude-3-5-sonnet`, and similar) were fine-tuned on dialogue data using RLHF. They learned what a helpful, harmless reply looks like in a conversation. They understand the system / user / assistant role structure natively and generate proper conversational replies — not raw continuations. You send a `messages[]` list; they reply as the assistant.

Chat models are trained to be a helpful conversation partner, not just a text predictor.

From an API perspective, the two endpoints look different. The older `/v1/completions` endpoint took a single prompt string. The modern `/v1/chat/completions` endpoint takes a `messages[]` list. OpenAI has deprecated the pure completion endpoint for its flagship models. For all new projects, use the chat completions endpoint — even for tasks that are not conversational.

Use `/v1/chat/completions` for everything — even single-turn, non-conversational tasks.

For single-turn tasks (summarize a document, classify a label), the difference is small: put your task in the user message. For multi-turn conversations, the chat structure is essential — the model maintains its persona and understands turn order. For building agents or fine-tuning, chat models are the standard. Almost all modern tooling assumes the chat format.

New projects → chat completions API, regardless of whether the task involves conversation.

Code examples

Legacy completion endpoint (shown for reference — do not use for new projects)

from openai import OpenAI
client = OpenAI()

# Old way: single string, model continues it
response = client.completions.create(
    model='gpt-3.5-turbo-instruct',
    prompt='The capital of France is',
    max_tokens=5,
)
print(response.choices[0].text)  # ' Paris.'

Shown for context only. The `/v1/completions` endpoint is deprecated for GPT-4 and later models. If you see old blog posts using this shape, translate them to the chat format shown below.

Modern chat completions endpoint — use this for all new projects

from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
    model='gpt-4o',
    messages=[
        {'role': 'system', 'content': 'Answer factually and briefly.'},
        {'role': 'user',   'content': 'What is the capital of France?'},
    ],
)
print(response.choices[0].message.content)  # 'Paris.'

Even for a single factual question, use the chat format. It gives you the system message lever, consistent structure across all tasks, and access to all current model features such as tools, structured output, and vision.

Common mistakes

Glossary

completion model
An older style of AI that works like a giant autocomplete — it continues whatever text you give it, without any concept of a conversation.
chat model
A modern AI fine-tuned to act as a conversation partner. It understands whose turn it is to speak and how to give a helpful, structured reply.
RLHF
Short for Reinforcement Learning from Human Feedback — a training method that teaches AI how to behave like a helpful, harmless assistant.

Recall questions

Questions & answers

A tutorial from 2021 uses `openai.Completion.create()` with a raw prompt string. You are adapting it for `gpt-4o`. What changes?

Switch from `client.completions.create(prompt='...')` to `client.chat.completions.create(messages=[...])`. Move the instructions into a system message and the input into a user message. The response field also changes: `response.choices[0].text` becomes `response.choices[0].message.content`.

Approach: This is a direct API translation. The key changes are the endpoint, the input shape (string → messages list), and the output field.

You need to build a pipeline that labels support tickets into categories. Should you use the chat completions API or the completions API? Why?

Use the chat completions API. Even though this is not a conversation task, the chat format is the modern standard for all tasks. Put classification instructions in the system message and the ticket text in the user message. The completions endpoint is deprecated for flagship models and does not support tools or structured output.

Approach: Reinforce that 'chat' does not mean conversational — it is just the API format for all GPT-4-class tasks.

Continue learning

Previous: System / User / Assistant Roles

Next: Chat Completions API

Return to AI Engineering Roadmap