Hallucination

Hallucination is the most well-known failure mode of large language models: generating text that sounds confident, fluent, and authoritative while being factually incorrect or entirely fabricated. The term "hallucination" captures the essential property — like a human hallucination, the AI's output feels real and coherent from the inside even when it diverges from external reality. LLMs hallucinate because they are fundamentally next-token prediction systems: at each generation step, the model predicts the most statistically likely continuation of the text, based on patterns learned from training data. When the question falls outside reliable training knowledge, the model generates what "sounds right" statistically rather than what is factually accurate. Common hallucination patterns include: citing academic papers that don't exist, inventing plausible but fake statistics, confidently stating outdated information as current, and filling in specific details (names, dates, URLs) with invented content that fits the expected format.

Hallucination severity varies significantly by task and domain. Models are relatively reliable for well-covered, widely-documented topics where training data is abundant and consistent. They become unreliable for specific claims about recent events (past training cutoff), niche domain expertise (uncommon medical procedures, obscure legal jurisdictions), specific numerical data (exact figures, statistics, pricing), and any factual claim that depends on specific source material the model may not have seen or may have seen in contradictory forms. The confidence with which models hallucinate is particularly dangerous — a model that prefaced uncertain statements with "I'm not sure, but..." would be far less harmful than one that states fabricated information in the same authoritative tone as verified facts.

For B2B teams deploying AI in customer-facing or decision-critical contexts, managing hallucination is a core reliability engineering challenge. The primary mitigation is grounding through RAG: providing verified source documents and instructing the model to answer only from those sources. Source citation requirements reduce hallucination by making the model "show its work" and making fabricated citations easier to catch. Human review workflows for high-stakes outputs add a verification step before AI-generated content reaches consequential decisions. Model evaluation suites that regularly test for hallucination on known-correct answers catch regressions when model versions or prompts change. Designing for hallucination — treating it as a known failure mode to be mitigated rather than an exceptional bug — is the realistic approach to deploying AI reliably in production.