Abstract: Scientific machine learning (SciML) is the burgeoning field combining scientific knowledge with machine learning for data-efficient predictive modeling. We will introduce SciML as the key to effective learning in many engineering applications, such as improving the fidelity of climate models to accelerating clinical trials. This will lead us to the question on the frontier of SciML: what about stiffness? Stiffness is a pervasive quality throughout engineering systems and the most common cause of numerical difficulties in simulation.
We will see that handling stiffness in learning, and thus real-world models, requires new training techniques. We will showcase how learning accurate models of battery degradation and the energy efficiency of buildings fails with previous techniques like physics-informed neural networks but succeeds with new stiffly-aware architectures like continuous-time echo state networks. This deep understanding of numerical issues in learning will feedback into traditional machine learning where we showcase how regularizing stiffness in a neural ODEs can halve the training time for image classification tasks. Together the audience will leave with a firm understanding of the role stiffness will play in the next decade of SciML.