Towards Out of Distribution Generalization in Mechanics

There has been an explosion of recent work in applying data driven methods to problems in mechanics, with particular emphasis on designing materials with novel functionality and predicting their properties. While traditional machine learning (ML) methods have enabled such breakthroughs, they rely on the assumption that the training (observed) data and testing (unseen) data are independent and identically distributed (i.i.d). However, when applied to real world mechanics problems with unknown test environments, a well-trained ML model is very sensitive to data distribution shifts and can break down when evaluated on the shifted test dataset. In contrast, out-of-distribution (OOD) problems assume that the data in test environments is allowed to shift, and many methods both within and outside the mechanics community, have been proposed to improve the out-of-distribution (OOD) generalization for ML methods. However, because most of these OOD methods have been developed for classification problems, and further because of the lack of benchmark datasets for OOD regression problems, the efficiency of these methods on regression problems, which are more important than classification for mechanics, is unknown. Therefore, in this work, we perform a fundamental study of OOD generalization methods for regression problems in mechanics. Specifically, we identify three OOD generalization problems: covariate shift, mechanism shift, and sampling bias, and develop two benchmark datasets for each problem based on the well-known MNIST and EMNIST-Letters datasets to investigate the performance of the OOD generalization methods. Our experiments show that for most cases, while the OOD algorithms perform better compared to traditional ML methods on OOD generalization problems in mechanics, there is a compelling need to develop more robust OOD methods that can generalize notions of invariance across multiple OOD scenarios. Overall, we expect that the combination of this study, as well as the benchmark datasets we developed, will enable further development of OOD methods for regression problems.