Skip to main content
Seminar | Mathematics and Computer Science Division

Partitioning a Large Simulation as It Runs

LANS Seminar

Abstract: As computer simulations continue to grow in size and complexity, they present a particularly challenging class of big data problems. Many application areas are moving toward exascale computing systems, systems that perform a billion billion (not a typo!) floating point operations per second (FLOPS).Simulations at this scale can generate output that exceeds both the storage capacity and the bandwidth available for transfer to storage, making post-processing and analysis challenging. One approach is to embed some analyses in the simulation while the simulation is running — a strategy often called in situ analysis — to reduce the need for transfer to storage. Another strategy is to save only a reduced set of time steps rather than the full simulation. In the latter case, the selected time steps are typically evenly spaced, where the spacing can be defined by the budget for storage and transfer. This strategy is easy to implement but fails to recognize interesting” regions of the simulation where additional saves might provide scientific value. Our work combines both of these ideas to introduce an online in situ method for identifying a more compelling set of time steps of the simulation to save. Our approach significantly reduces the data transfer and storage requirements, and it provides improved fidelity to the simulation to facilitate post-processing and reconstruction. We illustrate the method by using a computer simulation that supported NASA’s 2009 Lunar Crater Observation and Sensing Satellite mission.