Data-Driven Mixed Precision: Finding High-Error Hotspots with Low-Level Virtual Machine Instrumentation

Abstract: Mixed precision is becoming increasingly important in high performance computing (HPC), but deciding where lower precision is safe remains a difficult numerical and software-engineering problem. In this talk, we present a compiler-driven approach for tracking rounding error at source-line granularity and using that information to guide mixed-precision decisions. The method relies on low-level virtual machine-based instrumentation to recompute floating-point operations in higher precision at runtime, estimate absolute and relative rounding error and propagate that information through an application’s execution.

This enables a data-driven workflow for identifying high-error hotspots and other numerically sensitive operations, promoting only the arithmetic that matters and avoiding blanket precision conversions. The resulting workflow can be easily streamlined using artificial intelligence agents. Beyond small kernels, we also discuss the engineering needed to apply this kind of analysis to libraries as complex as Hypre and PETSc, including message passing interface-aware propagation and tracking error across function boundaries.

Bio: Ignacio Lagunais leader of the Parallel Systems Group and a computer scientist in the Center for Applied Scientific Computing at Lawrence Livermore National Laboratory in California. He is also a co-organizer of the International Workshop on Software Correctness for HPC Applications. Ignacio has received several best paper awards for his research, including two Hans Meuer Awards for Best Research Paper at the International Supercomputing Conference in 2019 and 2023.

Series: See upcoming and previous presentations at CS Seminar Series.