Ten Is Not a Hundred

A study tested various hallucination detection methods on chatbot responses with subtle numerical errors, finding that most detectors, including LLM judges and entailment models, fail to reliably identify these mistakes at production scale. The best-performing small fact-checker achieved only moderate accuracy, highlighting challenges in detecting minor factual discrepancies in AI-generated text.