Abstract: Today’s scientific simulations and advanced instruments are producing extremely large amounts of data every day, introducing significant burdens in data storage and transferring at runtime. SZ (the R&D100 2021 award winner) is an efficient error-bounded lossy compressor developed for scientific datasets. SZ offers different types of error controls such as absolute error bound, relative error bound and peak signal to noise ratio (PSNR). SZ supports diverse execution environments including CPU, openMP and GPU. SZ has been integrated in multiple I/O libraries such as HDF5 and ADIOS, as well as various scientific packages such as cosmology simulation package. SZ is a modular composable framework allowing users to customize different compression pipelines according to specific requirement or data characteristics.

In this presentation, I will present the latest research work and software development in the regard of the SZ compression framework, especially about diverse fresh compression methods which can adapt to a wide range of use-cases (e.g., ultra-high speed, high compression ratio or high fidelity on specific quality metrics) in practice.