Home/Glossary/AI hallucination
Glossary · Artificial intelligence

What is AI hallucination?

Also called: Hallucination · Confabulation · AI fabrication

Definition

An AI hallucination is a response from a generative AI model that is presented fluently and confidently but is factually wrong, unsupported by the given sources, or invented, such as a fake citation, an incorrect figure or a policy that does not exist. Hallucinations happen because language models generate statistically likely text rather than looking up verified facts. They can be reduced with grounding techniques such as retrieval-augmented generation, clear instructions and evaluation, but not eliminated, so human verification remains necessary.

A language model produces each token based on patterns learnt in training and the context it is given. When the prompt asks about something the model has little reliable information on, or the retrieved context is missing or ambiguous, it may still produce a fluent answer. Fluency is not evidence of accuracy, which is why hallucinations are easy to miss.

At work, hallucinations create real risk: invented case law in a legal draft, a wrong tax rate in a finance note, or a misquoted HR policy sent to an employee. The risk is highest where output is published, sent to customers or used for decisions without review.

Practical safeguards include giving the model the source material and asking it to answer only from that, requesting citations and checking them, allowing the model to say it does not know, using RAG for factual systems, and evaluating outputs on test sets. Training people to verify specifics such as names, numbers, dates and references is the most important control.

Key points

  • Confident, fluent output that is false or unsupported.
  • Caused by generating likely text rather than retrieving facts.
  • Reduced by grounding, citations and evaluation.
  • Cannot be fully eliminated today.
  • Verify names, numbers, dates and references before use.

An example at work

A finance associate in Bengaluru asks an AI tool for the GST rate on a service; it answers confidently with the wrong rate, which she catches by checking the official notification before the invoice goes out.

More from the Bodhih family

Bodhih Training Solutions, Bengaluru · Updated · All 51 terms
Common questions

More about AI hallucination

Why do AI models hallucinate?

Language models are trained to produce plausible continuations of text, not to check facts. When they lack reliable information, or the prompt pushes them to answer anyway, they generate text that fits the pattern of a correct answer without being one. Missing or poor-quality context in a RAG system has the same effect.

How can you reduce AI hallucinations?

Provide source material and instruct the model to answer only from it, ask for citations, allow it to say it does not know, break complex tasks into steps, and use retrieval-augmented generation for factual applications. Then verify key facts yourself and, for production systems, run regular evaluations on real questions.