Automated Defect Detection in Electron Microscopy of Radiation Damage in Metals

<br/>In this talk, we discuss our recent work on automating detection of defects in electron microscopy images of irradiated metals relevant for advancing the capabilities of in situ defect characterization of nuclear materials [1–5]. Radiation response of materials is a critical design constraint for future nuclear fission and fusion materials. Electron microscopy is widely used to explore defects in crystal structures, but human tracking of defects can be time-consuming, error prone, unreliable, and is not scalable to large numbers of images or real-time analysis. In this work, we discuss application of machine learning approaches to find the location and geometry of different defects in irradiated alloys, such as dislocation loops, black dot interstitial clusters, and cavities. We explore multiple deep learning methods and generally find performance approaching or equivalent that of human accuracy. We explore multiple avenues of assessment, including canonical classification metrics like precision and recall of specific defect identification (F1≈0.8), accuracy of microstructurally-relevant defect properties (e.g., average defect areal density and size distribution (≈10% fractional errors), and accuracy of macroscopic engineering properties such as hardening (10-20 MPa errors, or about 10% of total hardening) and swelling (≈0.3% swelling errors). These results suggest that specific images can have significant errors, but averaging over many images yields quite good results. We explore convergence of the results with number of training samples, finding that certain defect types are significantly less well detected, likely due both to their having reduced sampling and greater variability, as well as labeling and model limitations for small size features. Finally, we discuss recent efforts using synthetic data to improve object detection model training and reduce the need for a large corpus of labeled experimental data. We explore both deep learning approaches of image generation (namely variational autoencoders) to create synthetic dislocation loop images and physics-based electron dynamics simulations to create synthetic cavities. We generally find that the use of synthetic data is a practical path toward generating new training data essentially instantaneously, leading to improvements in objection detection model performance.<br/><br/>1. Shen, M.; Li, G.; Wu, D.; Yaguchi, Y.; Haley, J. C.; Field, K. G.; Morgan, D.; Ridge, O.; Ridge, O. A Deep Learning Based Automatic Defect Analysis Framework for In-Situ TEM Ion Irradiations. Comput Mater Sci 2021, 197 (November 2020), 110560. https://doi.org/10.1016/j.commatsci.2021.110560.<br/>2. Jacobs, R.; Shen, M.; Liu, Y.; Hao, W.; Li, X.; He, R.; Greaves, J. R. C.; Wang, D.; Xie, Z.; Huang, Z.; Wang, C.; Field, K. G.; Morgan, D. Performance and Limitations of Deep Learning Semantic Segmentation of Multiple Defects in Transmission Electron Micrographs. Cell Rep Phys Sci 2022, 100876. https://doi.org/10.1016/j.xcrp.2022.100876.<br/>3. Field, K. G.; Jacobs, R.; Shen, M.; Lynch, M.; Patki, P.; Field, C.; Morgan, D. Development and Deployment of Automated Machine Learning Detection in Electron Microcopy Experiments. Microscopy and Microanalysis 2021, 27 (S1), 2136–2137. https://doi.org/10.1017/s1431927621007704.<br/>4. Jacobs, R.; Patki, P.; Lynch, M. J.; Chen, S.; Morgan, D.; Field, K. G. Materials Swelling Revealed through Automated Semantic Segmentation of Cavities in Electron Microscopy Images. Sci Rep 2023, 13 (1). https://doi.org/10.1038/s41598-023-32454-2.<br/>5. Jacobs, R. Deep Learning Object Detection in Materials Science : Current State and Future Directions. Comput Mater Sci 2022, 211 (May), 111527. https://doi.org/10.1016/j.commatsci.2022.111527.