Abstract: Large language models often guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty. Such “hallucinations” persist even in state-of-the-art systems. We analyze this phenomenon from a mathematical perspective and find that the statistical pressures of the next-word prediction training pipeline induce hallucinations and current evaluation procedures reward guessing over acknowledging uncertainty. We also propose “open rubric” evaluations with explicit error penalties, providing a practical path to reliable LLMS by aligning their incentives. The talk will be fact-based, and the speaker will readily admit ignorance.
Joint work with Adam T. Kalai, Ofer Nachum and Eddie Zhang (Nature, 2026).
Bio: Santosh Vempala is Frederick Storey II Chair of Computing and Distinguished Professor in the School of Computer Science, with courtesy appointments in the schools of Mathematics and Industrial and Systems Engineering (ISyE). He served as the founding director of the Algorithms and Randomness Center and ThinkTank (2006-2011), and is currently the director of GT’s oldest interdisciplinary PhD program --- Algorithms, Combinatorics and Optimization. His research interests include algorithms, randomness, high-dimensional geometry, brain and computing-for-good (C4G). He graduated from CMU in 1997 advised by Avrim Blum and was on the MIT faculty until 2007 except for a year as a Miller Fellow at UC Berkeley. Vempala is also a Sloan, Guggenheim, ACM, AMS and generally excitable Fellow, especially when a phenomenon that appears complex from one perspective turns out to be simple from another. In recent years, he has been trying to understand the limits of sampling and optimization algorithms and building a computational theory of brain.
Series: See all upcoming talks at the LANS Seminars page.