How can we detect and reduce hallucinations?

Detection catches the lie; reduction makes it rarer.

Picture it first

If your friend tells a surprising story, you can check it: 'Show me the photo!' (citation), 'Tell me again tomorrow' (if details change, something's off), or ask someone else who was there (fact check). And to stop wild stories in the first place, you say 'Only tell me what you saw yourself'.

Now the same idea, in simple QA words

We fight hallucinations in two ways. Reduce them: give the AI the real documents (RAG), tell it 'answer only from these and say I don't know if it's missing', and ask it to show sources. Catch them: check each fact in the answer against the source, ask the same question a few times (if facts keep changing, be suspicious), and block answers that can't be supported. You can't get to zero, but you can measure and keep the rate low.

How the story connects to the real thing

In the storyIn the AI system
'Show me the photo!'Asking the AI for sources
'Tell me again tomorrow'Asking several times to check consistency
Asking someone else who was thereChecking facts against a trusted source
'Only tell me what you saw yourself''Answer only from the given documents'

Real-world example

A health-insurance claims assistant must never invent coverage. Reduction: answer only from the retrieved policy, cite the clause, temperature 0. Detection: every answer passes a groundedness check that splits it into claims and verifies each against the cited clause; if any claim is unsupported, the answer is replaced by 'Let me connect you to an agent'. Hallucinations caught in production drop sharply and are logged for the golden set.

Step by step, in plain words

  1. Give the AI real documents to answer from.
  2. Instruct: use only these, cite them, say 'I don't know' if missing.
  3. Generate the answer.
  4. Check each fact in the answer against the documents.
  5. Block or soften answers that fail the check.
  6. Add every caught mistake to your test set.

How a tester tests it

  • Measure hallucination rate on a fixed set (including unanswerable questions) before and after each mitigation.
  • Validate citations: does the cited chunk really contain the claim?
  • Check the 'I don't know' path works and isn't triggered on easy questions (over-refusal).
  • Calibrate the detector itself against human labels.

Try it yourself

1) Take a chatbot answer, split it into individual claims, and mark each claim as supported or unsupported by a source you trust. 2) Re-ask the same question 3 times and compare the specific facts.

Interview angle

“Can hallucinations be eliminated completely?”

No — they can be reduced and detected, not guaranteed away. So I design layered defences: grounding, instructions, verification, gating and monitoring, and I report a measured hallucination rate with thresholds per risk level, rather than promising zero.