Overcoming Data Scarcity in Materials Science with Meta-Learning

Machine learning (ML) has enabled materials scientists to rapidly screen vast search spaces of chemistry and structure, often uncovering complex relationships beyond the reach of simple analytic expressions¹. While a density functional theory calculation for a single material and property may take days or weeks, an ML model, once trained, can output thousands of materials property predictions in seconds. However, the small dataset size common in materials research has rendered more expressive ML models (necessary for approximating complex maps of structure to property) often prone to overfitting². Thus, ML in materials science has largely been limited to a small number of data-abundant materials properties. Yet, all structure-property relationships are unified by fundamental rules of physics and chemistry. So, information learned from training an ML model to predict one property should be useful for predicting other properties. Using this fact, several works have applied transfer learning to improve materials property prediction with less training data^3–5. Meta-learning, a.k.a. learning how to learn, is a recently resurgent ML paradigm which, like transfer learning, addresses the challenge of few-shot learning (learning with few training examples). While transfer learning leverages information between only two tasks, meta-learning pre-trains an ML model on a distribution of related tasks. Meta-learning has greatly outperformed transfer learning in few-shot image classification⁶, already achieving accuracies over 99% on some benchmark datasets when given just one training example per image class⁷. In materials science, meta-learning has been used to predict experimental hydrogen adsorption loading on nanoporous materials as a function of pressure and temperature⁸. Here, we demonstrate the first application of meta-learning to predict inorganic crystalline materials properties using atomic structure as input. We find that a Crystal Graph Convolutional Neural Network⁹ can simultaneously meta-learn formation energies, bandgaps, phonon density of states peak positions, and average shear and bulk moduli. We plan to apply our framework to data scarce materials properties which have evaded ML approaches, like semiconductor dopability limits, surface properties, and highly accurate formation energies computed by quantum Monte Carlo. Since our meta-learning framework is agnostic to the choice of ML model, its performance will consistently improve as newer architectures are developed.<br/>References<br/>1. Butler, K. T., Davies, D. W., Cartwright, H., Isayev, O. & Walsh, A. Machine learning for molecular and materials science. <i>Nat. 2018 5597715</i> <b>559</b>, 547–555 (2018).<br/>2. Dunn, A., Wang, Q., Ganose, A., Dopp, D. & Jain, A. Benchmarking Materials Property Prediction Methods: The Matbench Test Set and Automatminer Reference Algorithm. <i>npj Comput. Mater.</i> <b>6</b>, (2020).<br/>3. Chen, C., Ye, W., Zuo, Y., Zheng, C. & Ong, S. P. Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals. <i>Chem. Mater</i> <b>31</b>, 2020 (2019).<br/>4. Cubuk, E. D., Sendek, A. D. & Reed, E. J. Screening billions of candidates for solid lithium-ion conductors: A transfer learning approach for small data. <i>J. Chem. Phys</i> <b>150</b>, 214701 (2019).<br/>5. Lee, J. & Asahi, R. Transfer learning for materials informatics using crystal graph convolutional neural network. <i>Comput. Mater. Sci.</i> <b>190</b>, (2021).<br/>6. Finn, C., Abbeel, P. & Levine, S. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. in <i>International Conference on Machine Learning</i> (2017).<br/>7. Bertinetto, L., Henriques, J., Torr, P. H. S. & Vedaldi, A. Meta-learning with differentiable closed-form solvers. in <i>ICLR</i>(2019).<br/>8. Sun, Y. <i>et al.</i> Fingerprinting diverse nanoporous materials for optimal hydrogen storage conditions using meta-learning. <i>Sci. Adv.</i> <b>7</b>, (2021).<br/>9. Xie, T. & Grossman, J. C. Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties. <i>Phys. Rev. Lett.</i> <b>120</b>, (2018).