Automated Defect Detection in Electron Microscopy of Radiation Damage in Metals

In this talk, we discuss our recent work on automating detection of defects in electron microscopy images of irradiated metals.^1-5 Radiation response of materials is a critical design constraint for future nuclear fission and fusion materials. Electron microscopy is widely used to explore defects in crystal structures, but human tracking of defects can be time-consuming, error prone, unreliable, and is not scalable to large numbers of images or real-time analysis. In this work, we discuss application of machine learning approaches to find the location and geometry of different defects in irradiated alloys, such as dislocation loops, black dot interstitial clusters, and cavities. We explore multiple deep learning methods (Faster Regional Convolutional Neural Networks (Faster R-CNN), Mask R-CNN, and You Only look Once (YOLO)) and generally find performance approaching or equivalent that of human accuracy. We explore multiple avenues of assessment, including precision and recall of specific defect identification (F1[endif]--&gt;0.8), accuracy of microstructurally relevant averages (e.g., average defect areal density and size distribution ([endif]--&gt;10% fractional errors), and accuracy of modeled defect dependent macroscopic properties such as hardening (10-20 MPa errors, or about 10% of total hardening), and swelling ([endif]--&gt;0.3% swelling errors). These results suggest that specific images can have significant errors, but averaging over many images yields quite good results. We explore convergence of the results with number of training samples, finding that certain defect types are significantly less well detected, likely due both to their having reduced sampling and greater variability, as well as model limitations for small size features. In addition, our targeted evaluation tests also suggest the best path toward improving future models is not expanding existing databases with more labeled images but instead data additions that target weak points of the model domain, such as images from different microscopes, imaging conditions, irradiation environments, and alloy types. Finally, we discuss the new types of science enabled by the massive data which can be extracted with these approaches, e.g., detailed defect evolution during in-situ irradiations. We will also point to provided cyberinfrastructure to enable use of our models, including Python notebooks tailored for running on Google Colab and cloud accessible models on the Foundry for data, models and science,⁶ which enables inference on new images using only two lines of python code.<br/><br/>References:<br/>[1] W. Li, K. G. Field, and D. Morgan, “Automated defect analysis in electron microscopic images,” <i>npj Comput. Mater.</i>, vol. 4, no. 1, pp. 1–9, 2018, doi: 10.1038/s41524-018-0093-8.<br/>[2] M. Shen <i>et al.</i>, “Multi Defect Detection and Analysis of Electron Microscopy Images with Deep Learning,” <i>Accept. Publ. Comput. Mater. Sci.</i>, 2021.<br/>[3] M. Shen <i>et al.</i>, “A deep learning based automatic defect analysis framework for In-situ TEM ion irradiations,” <i>Comput. Mater. Sci.</i>, vol. 197, no. November 2020, p. 110560, 2021, doi: 10.1016/j.commatsci.2021.110560.<br/>[4] K. G. Field <i>et al.</i>, “Development and Deployment of Automated Machine Learning Detection in Electron Microcopy Experiments,” <i>Microsc. Microanal.</i>, vol. 27, no. S1, pp. 2136–2137, 2021, doi: 10.1017/s1431927621007704.<br/>[5] R. Jacobs <i>et al.</i>, "Performance, Successes and Limitations of Deep Learning Semantic Segmentation of Multiple Defects in Transmission Electron Micrographs", arXiv preprint arXiv:2110.08244, 2021.<br/>[6] R. Chard, <i>et al.</i>, "DLHub: Model and data serving for science", <i>Proc. - 2019 IEEE 33rd Int. Parallel Distrib. Process. Symp. IPDPS 2019</i> pp. 283–292, 2019. doi:10.1109/IPDPS.2019.00038.; U. of Chicago, U. of Wisconsin-Madison, "Foundry Materials Informatics Environment"(2021). https://ai-materials-and-chemistry.gitbook.io/foundry/v/docs/.