From algorithms to software: Making optimal experimental design easier and cheaper

Optimal experimental design (OED) is a method for getting the most useful information from experiments while performing as few experiments as possible to achieve a desired objective.  

It involves (1) defining a goal, (2) assigning a ​“utility function” that quantifies the value of different experimental outcomes and (3) balancing factors such as cost and time to yield the highest utility value. In ptychography, for example, scientists use a focused X-ray beam to scan across a sample, recording multiple overlapping patterns between adjacent areas and reconstructing the sample’s image in far greater detail than possible with conventional microscopy. OED can help materials scientists determine optimal exposure time of a sample at each scan position, avoiding overexposure that could damage a sensitive sample. 

So why wouldn’t scientists always use an optimal experimental design approach? Sometimes, generating an optimal design can be computationally complex, especially for large experiments; a scientist without specialized training might prefer a simpler, more familiar method. Also, if the relationship between factors is uncertain, a design that’s optimal for one model might perform poorly for another. And practical constraints such as limited time or resources might force scientists to settle for ​“good enough.”  

To address these challenges, researchers at the U.S. Department of Energy’s Argonne National Laboratory and their university partners have been developing tools to make OED more practical and reliable. Their focus has been on inverse OED, in which one tries to determine unknown parameters of a system by looking at indirect observations or measurements. OED is useful in many scientific applications, ranging from designing new materials to improving medical imaging.  

Highlighted here are four recent studies in OED by Argonne researchers.  

Using smart sampling for sensor placement

Researchers tackled this question by building a mathematical model that formulates the sensor placement optimization problem as a stochastic optimization problem and solves it with a sampling-based method. They used an approach in which past knowledge about the parameters is combined with new experimental data to figure out the best sensor setup.  

 The results for multiple values of an uncertainty parameter are shown in Fig. 1. 

Publication: A. Attia, S. Leyffer, and T. Munson, ​“Roxbust A-optimal experimental design for Bayesian inverse problems,” SIAM/ASA Journal on Uncertainty Quantification 13(2) 2025, doi: 10.1137/24M1667543 

Dealing with more uncertainty

In a follow-up study, researchers from Argonne and North Carolina State University extended this work to handle more complex problems, including nonlinear systems and greater uncertainty. 

This new framework remains optimal even when conditions change, and it doesn’t require extra tuning to fit the training data — making the model easier to use. The approach was tested successfully with 64 possible sensor locations and eight uncertain parameters, as indicated in Fig. 2. 

“Most important, the approach is utility-agnostic,” said Ahmed Attia, a computational mathematician in Argonne’s MCS division. ​“This means that the method can work across various contexts without being tailored to a specific function or cost metric.”  

Publication: A. Chowdhary, A. Attia, and A. Alexanderian, ​“Robust optimal design of large-scale Bayesian nonlinear inverse problems,” doi:10.48550/arXiv.2409.09137 

Designing faster algorithms for placing sensors 

Another team of researchers from Argonne and North Carolina State worked on making the process faster. They designed new randomized algorithms that can quickly figure out where to place sensors even for large, complex experiments. 

The researchers borrowed ideas from a well-established problem in numerical linear algebra called ​“column subset selection” — identifying the best columns in a matrix — and applied it to finding the best sensors in an experiment. The resulting algorithms are computationally efficient and can run in parallel. They also don’t need much parameter tuning and come with strong theoretical guarantees.  

The team also devised a way to estimate data at locations where there aren’t any sensors (see Fig. 3). 

“This approach provides an inexpensive way to complete the data without having to solve the inverse problem explicitly,” said Srinivas Eswar, an assistant computational scientist in Argonne’s MCS division and first author of the paper reporting the study results.  

Publication: S. Eswar, V. Rao, and A. K. Saibaba, ​“Bayesian D-optimal experimental designs via column subset selection,” doi: 10.48550/arXiv.2402,16000 

Developing open-source software for testing OED methods

Given the many criteria, approaches and technologies, OED may seem overwhelming. To lighten the burden, researchers from Argonne, North Carolina State University and Oklahoma State University have created the software suite PyOED. 

Written in the popular language Python, PyOED enables developing, testing and benchmarking new methods for data assimilation and model-constrained OED. Using the package’s examples and test problems, researchers can experiment with standard and innovative OED technologies (see Fig. 4). 

“We envision PyOED as a first step,” said Attia, one of the software package developers. ​“It is open source and highly extensible: new models can be quickly implemented, and it can be easily combined with other user-defined routines.” 

Publication: A. Chowdhary, S. Ahmed, and A. Attia, ​“PyOED: An extensible suite for data assimilation and model-constrained optimal design of experiments,” ACM Transactions on Mathematical Software 50(2), 1-22, 2024, doi: 10.1145/3653071