Hallucination Mitigation
Scenario
Your customer service bot confidently tells a user: 'Yes, we offer a lifetime warranty on all products!' The user buys the product. You do not offer a lifetime warranty. Before you read on, predict: did the model 'lie', and how do you prevent this?
Think about how LLMs generate text — then read on to understand why hallucinations happen and how to minimize them.
Mental model
Hallucinations are not lies — they are the model's autocomplete guessing what sounds plausible. You reduce them by shrinking the space of allowed guesses.
LLMs do not have a database of facts. They have a statistical map of language. When they do not know the answer, they do not naturally say 'I don't know' — they predict the next most likely word that fits the pattern. To stop this, you constrain their environment: force them to read from provided context (RAG), explicitly instruct them to refuse ungrounded questions, and set temperature to 0 to remove creative randomness.
Deep dive
Models do not know when they are wrong. They are designed to be helpful — so if you ask 'What is our SLA for enterprise customers?' without providing the actual SLA document, the model will predict the most likely-sounding SLA percentage. This is hallucination. It is not a bug; it is next-word prediction working exactly as designed.
Models default to plausible-sounding answers rather than admitting ignorance.
The strongest defense is grounding. Inject the exact facts into the prompt and instruct the model to use ONLY that context. But providing context alone is not enough — you must explicitly forbid the model from using outside knowledge. An instruction like 'Answer ONLY using the provided context. Do not use any knowledge beyond what is given here' is essential.
Provide the facts AND explicitly forbid the model from using outside knowledge.
Even with context, a model may guess if the context does not contain the answer. Give it an explicit escape route. Tell it exactly what to say when the information is missing: 'If the answer is not in the provided context, reply exactly: I don't know.' This shrinks the allowed output space to a safe default instead of a hallucinated guess.
Give the model permission to fail gracefully with an explicit 'I don't know' instruction.
`temperature=0` removes randomness from word selection. A high temperature (0.8) encourages creative, diverse text — perfect for poems, terrible for facts. Temperature 0 forces the model to always pick the highest-probability next word. This does not guarantee zero hallucinations — the most probable word can still be wrong — but it removes creative guessing and makes output deterministic.
Set temperature=0 for factual tasks. It removes creative randomness but does not eliminate all hallucination.
For high-stakes pipelines, add a verification step. Before showing the answer to the user, have the model (or a smaller fast model) check that every claim in the generated answer can be traced to a quote in the source context. If verification fails, block the response or show a 'I couldn't verify this' fallback.
Pipeline: Grounding → Strict prompt + I-don't-know → temperature=0 → (optional) verification.
Code examples
Anti-hallucination system prompt
system_prompt = """
You are a strict, factual support assistant.
Answer the user's question using ONLY the provided context.
Context:
{retrieved_docs}
CRITICAL INSTRUCTIONS:
1. If the answer is not fully in the context, reply EXACTLY: "I don't know."
2. Do not use any knowledge beyond the context above.
3. Do not guess or infer.
"""Three elements work together: a strict boundary ('ONLY the context'), an explicit failure mode ('reply EXACTLY: I don't know'), and a prohibition on guessing. All three are needed — omitting any one increases hallucination risk.
Setting temperature=0 for factual tasks
from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
model='gpt-4o',
messages=[
{'role': 'system', 'content': system_prompt},
{'role': 'user', 'content': 'What is your refund policy?'}
],
temperature=0.0 # removes creative randomness for factual tasks
)
print(resp.choices[0].message.content)temperature=0 makes the model deterministic — the same prompt always produces the same output. This removes the random variation that can produce creative but incorrect answers. Note: it does not prevent hallucination when the model's highest-probability word is itself wrong.
Citation-first chain-of-thought to ground the output
citation_prompt = """
Answer the user's question using the provided context.
First, extract the exact quotes from the context that support your answer.
Format:
QUOTES:
- [quote 1]
- [quote 2]
ANSWER:
[Your final answer]
"""By writing the quotes first, the model attends to the source text before generating the final answer. This chain-of-thought heavily grounds the output because the model cannot easily write a quote that does not exist in the context.
Key points
- Grounding: Inject the exact facts in the prompt so the model reads instead of guessing from pre-trained weights.
- The explicit escape: Tell the model exactly what to say when the answer is absent — 'I don't know' — so it has a safe fallback.
- temperature=0: Turn off creative randomness for factual tasks. Output becomes deterministic but is not hallucination-proof.
- Citations first: Ask the model to quote the source before answering — this chains its attention to the facts before it writes.
Common mistakes
- Assuming temperature=0 eliminates hallucinations completely: temperature=0 makes the model deterministic — it will make exactly the same mistake every time. If the most statistically probable next word is factually wrong, the model will still produce it confidently. Grounding and the explicit 'I don't know' instruction are the primary defenses.
- Telling the model 'don't hallucinate': LLMs don't understand 'truth' vs 'hallucination'. Saying 'don't lie' is far less effective than saying 'use only the provided text, and reply I don't know if the text doesn't contain the answer'.
Glossary
- hallucination
- When an AI does not know the answer but generates a plausible-sounding, confident, and completely false response rather than saying 'I don't know'.
- temperature
- A setting that controls how creative the model is. Setting it to 0 forces the model to pick the most statistically probable next word, reducing creative guessing.
- grounding
- The technique of forcing the model to base its answers on specific text you provide in the prompt, so it cannot rely on its unreliable pre-trained memory.
Recall questions
- Explain why LLMs hallucinate, as if to a classmate.
- What is the purpose of setting temperature to 0 for factual tasks?
- How does asking the model to write quotes before answering reduce hallucinations?
- Your RAG bot retrieves the correct context but still adds unverified extra details to its answers. Predict two things you should check and fix.
Questions & answers
Your RAG system retrieves the correct context, but the model still adds extra, unverified details to the answer. How do you fix this?
Three fixes: set temperature=0 to remove creative variation. Tighten the system prompt to explicitly forbid using outside knowledge and mandate an 'I don't know' fallback. Ask the model to quote the supporting context before writing the final answer — this chains its attention to the source text.
Approach: Address both the API parameter (temperature) and the prompt constraints. Interviewers want to see you know how to lock down model behavior at multiple levels.
Is it possible for a model running at temperature=0 to hallucinate? Why or why not?
Yes. temperature=0 makes the model deterministic — it will always predict the same highest-probability next word. But if the prompt is ambiguous, the context is contradictory, or the model's pre-trained bias is strong enough, the most probable word is still factually incorrect. temperature=0 means the model makes the same mistake every time, not zero mistakes.
Approach: Dispel the myth that temperature=0 equals perfect accuracy. The fix for hallucination is grounding — not temperature alone.
Continue learning
Previous: Grounding with Retrieval