Argonne National Laboratory

Argonne's Blue Gene/P to host large cadre of INCITE researchers

By Angela HardinJanuary 17, 2008

ARGONNE, Ill. — Twenty research projects have been awarded more than 111 million hours of computing time at the Argonne Leadership Computing Facility (ALCF) at Argonne National Laboratory.

The awards are part of a competitively selected group of 55 scientific projects announced Thursday by the Department of Energy's Office of Science. The awards are made through the 2008 Innovative and Novel Computational Impact on Theory and Experiment (INCITE), a DOE program that supports computationally intensive, large-scale research projects. DOE has allocated more than 265 million processor-hours for supercomputing and data storage resources located at Argonne, Oak Ridge , Pacific Northwest and Lawrence Berkeley national laboratories.

"The Department of Energy's Office of Science has two of the top ten most powerful supercomputers, and using them through the INCITE program is having a transformational effect on America 's scientific and economic competitiveness," DOE Under Secretary for Science Raymond L. Orbach said. "Once considered the domain of only small groups of researchers, supercomputers today are tools for discovery, driving scientific advancement across a wide range of disciplines. We're proud to provide these resources to help researchers advance scientific knowledge and understanding and thereby to provide insight into major scientific and industrial issues."

At Argonne, new and returning INCITE researchers will conduct projects ranging from large-scale simulations of potentially dangerous heart rhythm disorders to running detailed numerical experiments of thermal striping in sodium-cooled fast reactors.

"It is thrilling to see the broad range of scientific projects to be conducted at Argonne ," Argonne Director Robert Rosner said. "The investigators on these projects will be able to conduct cutting-edge research that will take only a few weeks or months – a relatively short period of time compared to the years and decades that would have been needed without DOE's supercomputing resources. That means important scientific findings can be made more quickly and used to develop technologies t hat will benefit U.S. economic competitiveness and address society's concerns about the environment, clean and efficient energy, climate change, and healthcare, to name just a few."

"For example, Andrew Siegel, Argonne's project leader for nuclear simulation, will have the opportunity through INCITE to test a set of integrated models that will help to optimize and validate the design and safety of a new generation of advanced recycle reactors," Rosner said. "The integration of these models would represent a sea change over the traditional reliance on expensive instrumented experiments. In the long run, advanced simulation will significantly reduce the cost to construct new nuclear reactors, a carbon-emissions-free source of electricity."

ALCF Director Raymond Bair said that, " Argonne has expanded its supercomputing capabilities to give scientists even more advanced computing resources with which to conduct more detailed and accurate simulations of scientific problems. Within the last year, the ALCF has expanded its computing power by more than a factor of twenty, to 111 teraflops. Argonne 's recent order for a 445-teraflops IBM Blue Gene/P supercomputer will soon bring the ALCF a total computing power of 556 teraflops. A lot of important science is going to be done on this machine."

Of the 20 INCITE projects that will use the Blue Gene/P at Argonne , 13 are new projects and seven are projects renewed from 2007.

New projects:

  • David Dean of Oak Ridge National Laboratory (ORNL) was awarded 10 million hours at Argonne and 7.5 million hours at ORNL to calculate structural and reaction properties of light- and medium-mass nuclei into the iron region from the complex two- and three-nucleon forces among protons and neutrons, potentially providing an understanding from first principles to triple-alpha burning – the process by which a mature star creates carbon-12 nuclei from helium-4 nuclei and the structure of exotic nuclei with many more neutrons than protons.
  • Robert Sugar of the University of California-Santa Barbara was awarded 19.6 million hours at Argonne and 7.1 million hours at ORNL to generate gauge configurations with up, down and strange quarks on sufficiently fine-grained lattices that have sufficiently small up and down quark masses in order to enable the extrapolation of key quantities of the chiral and continuum limits. The gauge configurations will be used to determine a wide range of physical quantities that are important to high energy and nuclear physics research.
  • Robert Minnich of Sandia National Laboratories was awarded 1 million hours to scale the Plan 9 distributed operating system on Blue Gene/P, measure the performance of applications of interest, and test all aspects of the operating system's environment in preparation for future computer systems with 10 million central processing units.
  • Warren Washington of the National Center for Atmospheric Research was awarded, as principal investigator, 1 million hours at Argonne, 1.3 million hours at the National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory, and 15.7 million hours at ORNL to work with a large community of scientists to develop and use the next generation of DOE and National Science Foundation's Community Climate System Model.
  • Christopher Mundy of Pacific Northwest National Laboratory was awarded 750,000 hours at both Argonne and ORNL to develop a new understanding of chemical reactions in solutions and at interfaces, especially in the area of hydrogen storage and catalysis in order to establish a protocol for the application of high-performance computing to current and future Grand Challenges in the chemical sciences.
  • Christopher Wolverton of Northwestern University was awarded 1 million hours to rationally design novel nanostructured hydrogen storage materials with fast (de)hydrogenation kinetics and favorable thermodynamics.
  • David Baker of the University of Washington was awarded 12 million hours to accurately predict structures of biologically important proteins; engineer novel protein-protein interactions and protein-based inhibitors, which would be a significant step towards development of novel therapeutics; and to design catalysts of carbamate hydrolysis with applications in contaminated soil bio-remediation.
  • William Tang of Princeton University and the Princeton Plasma Physics Laboratory was awarded 2 million hours to gain a better understanding of turbulence as a primary mechanism by which particles and energy diffuse across the confining magnetic field in toroidal fusion systems. Results from these studies may have direct relevance to the future performance of the international burning plasma experiment called ITER.
  • Don Lamb of University of Chicago's ASC/Alliance Flash Center was awarded 21 million hours to conduct the first rigorous, systematic validation of four current models of the type Ia supernovae and to determine whether Rayleigh-Taylor-driven turbulent nuclear burning occurs primarily at large scales or small scales; and at what physical conditions the transition from the flamelet burning regime distributed burning regime takes place.
  • Thierry Poinsot of the European Centre for Research and Advanced Training in Scientific Computation was awarded 4 million hours for a first-time application of a new simulation method – called Large Eddy Simulation – to the computation of the unsteady reacting flow within a complete helicopter turbine chamber.
  • Andrew Siegel of Argonne was awarded 5 million hours to conduct detailed numerical experiments of thermal striping in sodium-cooled fast reactors. Designers of the Advanced Recycle Reactor will use the project's results to better understand the physics of jet mixing in reactor vessels, leading to more optimal designs for future facilities.
  • William George of the National Institute of Standards and Technology was awarded 750,000 hours to study the flow of dense suspensions and related colloidal systems composed of rigid bodies, with and without interparticle interactions, having a wide range of size and shape, and under a variety of flow conditions, such as shear and around obstacles.
  • Jeffrey Fox of Gene Network Sciences was awarded 846,720 hours to simulate potentially dangerous rhythm disorders of the heart that will provide greater insight into these disorders and an opportunity to test ideas for how to prevent or treat them.

In renewed endeavors:

  • Paul Fischer of Argonne was awarded 14 million hours to carry out first-principles-based simulation and analysis of reactor core cooling, which will provide insight to design improvements leading to increased safety and economy of advanced reactors.
  • Giulia Galli of the University of California-Davis (UCD) was awarded 6 million hours to conduct quantum simulations to investigate how water interacts with the surfaces of various materials and to study how the properties of liquid water change when confined in very small spaces.
  • Patrick H. Worley of ORNL leads a multi-institutional consortium of computer scientists awarded 4 million hours at both Argonne and ORNL to investigate the performance characteristics of leadership-class computing systems and to develop performance tools and methodologies for these systems. These data and tools will enable computational scientists and system administrators to use leadership-class computer systems more effectively and help them prepare to use the next generation petascale systems, accelerating the achievement of the INCITE science goals.
  • A Pratt & Whitney team led by Peter Bradley was awarded more than 1.3 million hours to develop improved aircraft engine combustor simulations to enable reduced emissions and improved operability.
  • Igor Tsigelny of the University of California-San Diego was awarded 1.2 million hours to study the mechanism of membrane-protein interactions and model membrane penetration dynamics of the synuclein oligomers and to compare these results with experimental findings in order to understand the molecular basis of Parkinson's disease and to generate leads for drug discovery.
  • Kelly Anderson of Proctor and Gamble was awarded 4 million hours to investigate the molecular mechanisms of surfactant-assisted bubble formation.
  • Benoit Roux of Argonne and the University of Chicago was awarded 1.5 million hours at Argonne and 3.5 million hours at Oak Ridge to investigate the voltage-gating mechanism of membrane ion channels in order to understand how the membrane-associated molecular protein-machines are able to carry out their functions.

Returning INCITE researchers tout the enabling research the program has already facilitated.

Igor Tsigelny of the University of California-San Diego wrote in a SciDAC Review article about his DOE-supported work that: "Using the IBM Blue Gene's considerable computer simulation power, researchers have made major strides in evaluating the molecular and environmental features that led to the clinical diagnosis of Parkinson's disease and Lewy body dementia, as well as to strategies for pharmaceutical intervention and amelioration of Parkinson's disease. Their findings also have broad applicability to other diseases such as Alzheimer's disease, rheumatoid arthritis, type II diabetes mellitus, and spongiform encephelopathies (prion diseases)."

Peter Bradley of Pratt & Whitney, said "The DOE's INCITE program has been a very valuable tool in developing Pratt & Whitney's next generation of green engine technology. Pratt & Whitney's INCITE award is enabling the development of new computer simulation techniques being used to design a new family of commercial aircraft engines that will deliver unprecedented fuel efficiency and double-digit reductions in greenhouse gas emissions."

Giulia Galli, a chemistry professor at the University of California-Davis said that, "Access to dedicated computer time on a high-performance platform is one of the key ingredients for the success of complex investigations. At ANL, we not only have access to a very powerful machine and dedicated cycles, but also to a very competent and efficient staff who are extremely helpful in facilitating our work and in fixing our problems as we encounter them. So we have quite a number of reasons to be grateful."

And David Baker, a biochemistry professor at the University of Washington , said that, "The INCITE program is making possible the exploration of whole new areas in computational structural biology."

New INCITE awardees, such as Jeffrey Fox, vice president of cardiovascular research for Gene Network Sciences, a privately held biosimulation company, are looking forward to the opportunity to use the Blue Gene/P facility. "We're excited about the INCITE award and the opportunity to use the Argonne machine," Fox said. "It will help us conduct large-scale simulations that we could not consider attempting without such a resource."

The next round for INCITE competition will be announced this summer. Expansion of the DOE Office of Science's computational capabilities should approximately quadruple the 2009 INCITE award allocations to close to a billion processor hours. This opportunity, available to all on a competitive basis, will lead to scientific discovery on an unprecedented scale.

Additional information:

To read the Department of Energy's INCITE announcement , please visit

For more information on the INCITE program, please visit