Machine learning enables systems to learn automatically, based on patterns in data, and make better searches, decisions, or predictions. Machine learning has become increasingly important to scientific discovery. Indeed, the U.S. Department of Energy has stated that ​“machine learning has the potential to transform Office of Science research best practices in an age where extreme complexity and data overwhelm human cognitive and perception ability by enabling system autonomy to self-manage, heal and find patterns and provide tools for the discovery of new scientific insights.”

What have we been doing?

In the Mathematics and Computer Science Division at Argonne, we are exploring various projects involving machine learning, ranging from algorithm and software development to applications in science and the environment. The following are some examples:

• Predictive modeling of wide area data transfer
• Study of aquifer systems
• Creation of a lightweight thermal prediction system for runtime management
• Parallel I/O optimization for scalable machine learning
• Detection of silent data corruption
• Prediction models of system performance and power
• Load balancing of climate models
• Novel algorithms for Bayesian and blackbox optimization
• Scalable frameworks for neural network hyperparameter optimization and tuning

Where have we been publishing?

Here are some recent papers we have published in peer-reviewed journals or presented at conferences.

New software

• MaLTESE -- Machine Learning Tool for Engine Simulations -- https://link.springer.com/chapter/10.1007%2F978-3-030-20656-7_10
• DeepHyper -- Scalable Neural Architecture and Hyperparameter Search for Deep Neural Networks -- https://​www​.alcf​.anl​.gov/​u​s​e​r​-​g​u​i​d​e​s​/​d​e​e​p​h​y​per-0

Outreach

We’ve also been involved in several outreach activities.

• 2023 Argonne Training Program on Extreme-Scale Computing (ATPESC), August 2023 Track 8: Machine Learning -- lecture by Marieme Ngon on Introduction to Deep Learning
• Workshop on Artificial Intelligence and Machine Learning for Scientific Applications. Author/Presenters: Automated Continual Learning of Defect Identification in Coherent Diffraction Imaging – Orcun Yiliz, Henry Chan, Krishnan Raghavan, William Judge, Mathew J. Cherukara, Prasanna Balaprakash, Subramanian Sankaranarayanan, Tom Peterka, SC22, 2022
• 2022 Argonne Training Program on Extreme-Scale Computing (ATPESC), August 2022 Track 2: Programming Models and Languages – MPI, Track 3: I/O, Track 5: Numerical Algorithms and Software for Extreme-Scale Science, Track 7: Software Productivity and Sustainability, Track 8: Machine Learning
• Argonne Training Program on Extreme-Scale Computing (ATPESC) 2020 – several lectures on machine learning, including autoencoders and hyperparameter optimization
• Panel at SC19: Machine Learning Hardware: Architecture, System Interfaces, and Programming Models
• Basic Research Needs Workshop for Scientific Machine Learning Core Technologies for Artificial Intelligence, 2019 (member, organizing committee)
• SciML2018: DOE ASCR Workshop on Scientific Machine Learning (member, organizing committee)
• Deep Learning Workshop - at Argonne in January 2018
• 2017 Summer Student Symposium - This past summer several several students pursued projects in machine learning, including study of a smart pipeline for urban data science and use of deep neural networks for wind speed forecasting.
• Machine Learning Workshop for students and postdocs - at Argonne in July 2017.
• Machine Learning Workshop at Argonne in March 2017 - MCS Division researchers gave presentations on topics including application performance prediction on HPC systems