AuroraGPT: Foundation Models for Science

The AuroraGPT project leverages the Aurora exascale supercomputer at the Argonne Leadership Computing Facility (ALCF) to develop and enhance understanding of powerful foundation models (FMs), such as large language models, for scientific discovery. The ALCF is a U.S. Department of Energy Office of Science user facility.

By creating FMs for science—while developing underlying capabilities, tools and workflows, data resources, and other processes and artifacts—the AuroraGPT project aims to significantly improve how science is conducted by fostering a deeper integration of AI capabilities into research workflows.

Scientists are creating and evaluating a series of increasingly powerful FMs, each with more parameters and/or trained on more data than those that precede it.  The models are designed to assist researchers in making more informed and efficient discoveries. The AuroraGPT research program focuses on producing this sequence of FMs while ensuring that each provides both a scientifically useful capability and knowledge concerning scientific and computational performance to guide the design of the next model in the sequence.

Intended outcomes from the project include the following:

• Datasets and data pipelines for preparing science training data
• Software infrastructure and workflows to train, evaluate, and deploy LLMs at scale for scientific research
• Evaluation of state-of-the-art LLM models to determine where they fall short in deep scientific tasks and where deep data may have an impact
• Assessment of the value of augmenting web training data with science-specific data (full-text scientific papers, structured scientific datasets)
• Research grade artifacts (models) for the scientific community and adaptation for downstream uses
• Promotion of responsible AI best practices
• International collaborations around the long-term goal of AI for science