Argonne participates in first simulation of fully coupled nuclear reactor core
Researchers from Argonne National Laboratory, together with colleagues from Pennsylvania State University, Oak Ridge National Laboratory and the University of Illinois at Urbana-Champaign, have conducted a simulation of the first fully coupled, fully resolved nuclear reactor core. The simulation, which achieved a sustained multiphysics performance on 9,000 nodes of the Frontier supercomputer, has been named a finalist for the 2023 Gordon Bell Prize.
A key component in achieving this transformative leap was the Argonne-developed NekRS solver.
A challenging problem
The ACM Gordon Bell Prize, which comes with a $10,000 award, is widely considered the premier prize in high-performance computing. The prize recognizes outstanding “innovation in applying high-performance computing to applications in science, engineering, and large-scale data analytics.”
To this end, in their paper “Exascale multiphysics nuclear reactor simulations for advanced designs,” the research team described their achievement in tackling one of the most challenging problems confronting physicists: modeling novel nuclear reactors without comprising in terms of operational margins.
In the simulations, NekRS was coupled with a high-fidelity Monte Carlo neutron transport code.
The coupled full-core simulations were conducted with ENRICO (for Exascale Nuclear Reactor Investigative Code), an application developed under the U.S. Department of Energy Exascale Computing Project. The ENRICO results presented in the paper are the first time both the fluid flow and Monte Carlo radiation transport solvers have executed on graphics processing units.
Figure 1 shows strong-scaling results for a problem involving 1.6 billion gridpoints. Such high-fidelity simulations were infeasible until recently. What makes NekRS so powerful? The answer lies in several innovations, including its rapidly convergent high-order spectral element discretizations, selection of the fastest kernel option for each configuration, optimized communication performance invoked in different precisions and for different polynomial orders, and characteristics-based advection enabling larger time steps.
“Simulations of full nuclear reactor cores are ushering in a new era for coupled thermal-fluids and radiational transport analysis of nuclear systems,” said Misun Min, a computational scientist in the Mathematics and Computer Science division at Argonne and one of the lead developers of NekRS. “We expect our work to provide engineers with new insights into the behavior of advanced nuclear reactor designs.”
A track record of innovation
Another important aim of the Gordon Bell Prize is to “track the progress over time of parallel computing.”
The researchers showed how the performance achieved by NekRS on the past three generations of top supercomputers has dramatically increased, culminating in the groundbreaking runs on 9,000 nodes of Frontier, the world’s first exascale supercomputer.
“Before NekRS, the largest pebble bed calculations were on the order of 1,000 pebbles, With NekRS, we have been able to achieve the first fully resolved full-core coupled calculations,” Min said.
Not only has the problem size that NekRS is able to simulate increased by a factor of 35, but the time required per timestep has also decreased.
“Thus, NekRS not only is executing substantially faster, it is also resolving more complex physical phenomena than previously possible,” Min said.
The team for the Gordon Bell Prize finalist project “Exascale Multiphysics Nuclear Reactor Simulations for Advanced Designs” comprises Elia Merzari, Steven Hamilton, Thomas Evans, Misun Min, Paul Fischer, Stefan Kerkemeier, Jun Fang, Paul Romano, Yu-Hsiang Lan, Malachi Phillips, Elliot Biondo, Katherine Royston, Tim Warburton, Noel Chalmers, and Thilina Rathnayake. The presentation will take place at the 2023 International Conference for High Performance Computing, Networking, Storage, and Analysis (SC23) on Tuesday, November 14, 2023, at 11:30am - 12pm MST.
To learn more about SC23 activities, see SC23 website: https://sc23.supercomputing.org/attend/schedule/.