Automated Defect Detection in Electron Microscopy of Radiation Damage in Metals

In this talk, we discuss our recent work on automating detection of defects in electron microscopy images of irradiated metals.^1-5 Electron microscopy is widely used to explore defects in crystal structures, but human tracking of defects can be time-consuming, error prone, and unreliable, and is not scalable to large numbers of images or real-time analysis. In this work, we discuss application of machine learning approaches to find the location and geometry of different defects in irradiated alloys, such as dislocation loops, black dot interstitial clusters, and cavities. We show that performance comparable to human analysis can be achieved with relatively small training data sets consisting of order one thousand labeled defects. We explore multiple deep learning methods (Faster Regional Convolutional Neural Networks (Faster R-CNN), Mask R-CNN, and You Only look Once (YOLO)) and generally find performance approaching or equivalent that of human accuracy. We explore multiple avenues of assessment, including precision and recall of specific defect identification (F1~0.8), accuracy of microstructurally relevant averages (e.g., average defect areal density and size distribution (~10% fractional errors), and accuracy of macroscopic properties dependent on number, size and type of defects present such as hardening (10-20 MPa errors, or about 10% of total hardening), and swelling (~0.3% swelling errors). These results suggest that averaging over many images can reduce the impact of errors. We explore convergence of the results with number of training samples, finding that certain defect types are significantly less well detected, likely due both to their having reduced sampling and greater variability. In addition, our targeted evaluation tests also suggest the best path toward improving future models is not expanding existing databases with more labeled images but instead data additions that target weak points of the model domain, such as images from different microscopes, imaging conditions, irradiation environments, and alloy types. Finally, we discuss the new types of science enabled by the massive data which can be extracted with these approaches, e.g., detailed defect evolution during in-situ irradiations.<br/><br/>References:<br/>[1] W. Li, K. G. Field, and D. Morgan, “Automated defect analysis in electron microscopic images,” <i>npj Comput. Mater.</i>, vol. 4, no. 1, pp. 1–9, 2018, doi: 10.1038/s41524-018-0093-8.<br/>[2] M. Shen <i>et al.</i>, “Multi Defect Detection and Analysis of Electron Microscopy Images with Deep Learning,” <i>Accept. Publ. Comput. Mater. Sci.</i>, 2021.<br/>[3] M. Shen <i>et al.</i>, “A deep learning based automatic defect analysis framework for In-situ TEM ion irradiations,” <i>Comput. Mater. Sci.</i>, vol. 197, no. November 2020, p. 110560, 2021, doi: 10.1016/j.commatsci.2021.110560.<br/>[4] K. G. Field <i>et al.</i>, “Development and Deployment of Automated Machine Learning Detection in Electron Microcopy Experiments,” <i>Microsc. Microanal.</i>, vol. 27, no. S1, pp. 2136–2137, 2021, doi: 10.1017/s1431927621007704.<br/>[5] R. Jacobs <i>et al.</i>, "Performance, Successes and Limitations of Deep Learning Semantic Segmentation of Multiple Defects in Transmission Electron Micrographs", arXiv preprint arXiv:2110.08244, 2021.