Next Generation Linear Algebra: One Singular Value Decompositions across Data Size, Precision and Hardware

Abstract: We present a portable, GPU-accelerated implementation of a QR-based Singular Value algorithm in Julia that allows code reproducibility across several different GPU vendors and code abstraction across data size, covering data sizes beyond GPU memory through an out-of-core algorithm.

Singular value decomposition is a fundamental numerical tool in scientific computing and machine learning, providing optimal low-rank matrix approximations with applications ranging from dimensionality reduction to data compression and signal processing. Our implementation leverages Julia’s multiple dispatch and metaprogramming capabilities, integrating with the GPUArrays and KernelAbstractions frameworks to provide a unified type- and hardware-agnostic functionality. It supports diverse GPU architectures and data types, including half-precision and Apple Metal.

We benchmark the algorithm against several state-of-the-art linear algebra libraries, demonstrating performance on par with, or in some cases exceeding, vendor libraries. We explore GPU kernel optimization through parameter tuning to enable efficient parallelism and improved memory locality. Performance results on multiple GPU backends and data types demonstrate scalability combined with reproducibility, highlighting Julia’s suitability for high-performance linear algebra in heterogeneous environments.

Bio: Evelyne Ringoot is a Ph.D. candidate in the departments of Mathematics and Computational Science at the Massachusetts Institute of Technology, advised by Professor Alan Edelman. She previously obtained an M.Sc. in Civil Engineering at the Vrije Universiteit Brussel/Université Libre de Bruxelles, was a visiting student at EPFL Lausanne and Ulsan UNIST and has industry experience in data analytics consulting.

See upcoming and previous presentations at CS Seminar Series.