AI Hallucination

In generative artificial intelligence, a hallucination is an output that appears fluent and plausible but contains false, fabricated, inconsistent, or unsupported information. It occurs because a generative model predicts likely content from learned patterns rather than independently guaranteeing truth. The phenomenon is also called confabulation in risk-management literature.

Why hallucinations occur

Generative models usually produce responses through probabilistic prediction, so linguistic plausibility does not necessarily imply factual accuracy. Hallucination is not deliberate deception because the model has no human-like intention to lie.

• Training data may be incomplete, outdated, inconsistent, or contain errors and biases.
• An ambiguous prompt or a request outside the model's learned information can encourage unsupported completion.
• Weak grounding in authoritative external data can cause the model to fill information gaps with plausible text.
• Complex reasoning, calculation, or long-context tasks can produce internally inconsistent outputs.

Common manifestations and risks

Hallucinations can take several observable forms and become especially serious when users mistake confidence or fluency for reliability.

• A model may invent facts, events, quotations, legal provisions, sources, or bibliographic citations.
• It may provide incorrect calculations, causal explanations, summaries, or interpretations.
• In high-stakes domains such as healthcare, law, finance, public administration, and scientific research, such errors can mislead decisions and weaken accountability.
• Fabricated content may also spread misinformation and undermine trust in digital systems.

Reducing the problem

Hallucination can be reduced but not assumed to be completely eliminated; safeguards should combine technical controls with human oversight.

• Ground outputs in trusted databases or documents, including through retrieval-augmented generation.
• Require citations, provenance, and independent verification of important claims.
• Use calculation engines, search tools, or domain-specific systems for tasks requiring exact results.
• Test models systematically, define acceptable-use boundaries, monitor failures, and retain qualified human review for consequential decisions.

How UPSC asks this