Reducing and Moving Light Source Data Intelligently

Why is data compression needed if a terabit streaming connection exists? 

Data compression is still needed because the data generation speed at advanced light source facilities exceeds the data transfer capacity between the facility and the supercomputing system.  

What are the limitations of current data compression methods? 

Both lossy and lossless compression methods are widely available, but they often are too slow or unable to distinguish the irregular regions typical of light source data. And techniques involving data partitioning and parallel processing across hundreds of nodes/cores, while producing promising compression ratios, are too expensive and may not scale efficiently. 

How are these limitations being addressed? 

To address these limitations, a team of scientists at Argonne, the University of Florida and the University of Iowa have developed lsCOMP, a compression technique that efficiently manages light source data within a single GPU kernel. Their research is reported in a paper published in SC’25: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2025. 

To evaluate lsCOMP, the team collected thousands of light source images from two key light source applications: 

• CSSI (Coherent Surface Scattering Imaging): Targets nanoscale static structures and is well suited for lossy compression methods. 

• XPCS (X-ray Photon Correlation Spectroscopy): Focuses on the dynamics of materials at the nanoscale, for which lossless compression is more desirable.   

For these applications, the team compared lsCOMP with several state-of-the-art GPU compressors on a single NVIDIA GPU: LZ4, ZSTD and Cascaded for lossless and cuZFP, TConv and ZThresh  for lossy compressors. Several features of lsCOMP are noteworthy.  

High throughput: Ideally, researchers want to achieve compression throughput that is higher than the data generation rate of the advanced light source. The results with lsCOMP were excellent. For example, it achieved throughput up to 20 times faster compared with lossless compressors. 

“Compression throughput is considered crucial for light source facilities, speeding up analysis and enabling real-time processing.,” said Yafan Huang, a visiting graduate student in Argonne’s Mathematics and Computer Science (MCS) division from the University of Iowa and first author of the paper. 

Data quality: The researchers also evaluated the quality of data compression. Again, the results were impressive. For example, with a of dataset consisting of 600  two-dimensional images and millions of random granular patterns (called speckles) common in light source images, lsCOMP preserved critical data features, while cuZFP had trouble distinguishing high- and low-frequency components, leading to reduced data quality. 

“Preserving data quality with compression is a key concern for scientists,” said Sheng Di, .a computational scientist in Argonne’s MCS division and Huang’s co-advisor ​“Different light source applications have different data quality requirements. lsCOMP showed superior performance in both visualization and speckle analysis.”