Hallucination
The phenomenon of AI generating plausible but false content
Hallucination is the phenomenon where LLMs generate factually incorrect information as plausible text. It is divided into Intrinsic Hallucination (content contradicting training data) and Extrinsic Hallucination (fabricating content not in training data). The cause lies in the LLM learning method itself. LLMs are trained to maximize "next token probability," not to "tell the truth." Because they generate the statistically most plausible next word, they cannot distinguish fact from fiction. It can be mitigated with RAG, Grounding, RLHF, etc., but complete resolution remains difficult.
Key Concepts
User asks about a specific fact
LLM searches for related patterns in training data (probabilistic generation, not exact recall)
If related information is insufficient, it "fabricates" statistically plausible content
Presents incorrect information with high confidence
Risk of user accepting without fact-checking
Mitigation methods: RAG (external document reference), Grounding (source citation), uncertainty expression ("it may be...")
Pros
- ✓ Understanding LLM limitations through hallucination research โ safer usage
- ✓ Motivation for advancing hallucination detection/mitigation technology
- ✓ Advancement of factuality benchmarks
Cons
- ✗ Serious problems in high-risk fields like medical/legal
- ✗ Difficult for users to detect hallucinations
- ✗ Cannot be completely eliminated with current technology
- ✗ Reduced LLM trustworthiness