Advancements in Real-Time Quantification for In-Situ Materials in Extreme Environments

In this talk, we discuss our recent developments on integrating automated detection of defects^1–5, stereomicroscopy^6,7, edge-computing⁸, and <i>in situ </i>ion irradiations to form a high-throughput, limited-bias workflow for studies on materials in extreme environments. Materials in extreme environments has been identified as a key research challenge for the materials community⁹, where radiation at elevated temperature, as well as stress, is one facet of the materials in extreme environments paradigm. Experiments on radiation-temperature effects is highly complex and dynamical, where defects are nucleated and can move in 1D and 3D with changes in their size, morphology, character, and can be annihilated. We discuss the application of machine learning to form a backbone for a high-fidelity, rapid quantification architecture that can perform 2D-projection and 3D visualization of defects and analysis of these complex dynamical events per frame and in near real-time fashion. We show that performance of deep learning methods, such as Mask R-CNN and You Only Look Once (YOLO), demonstrate performance levels that are at or exceed human performance with inference times below 100 ms. We see that such high-level model performance persists even when trained using physics-based synthetic datasets. We explore the application of deep learning coupled to tracking algorithms, such as DeepSORT, ByteTrack and BoT-SORT, to track hundreds of defects during extreme exposures, including during in-situ transmission electron microscopy (TEM) ion irradiations of several FeCrAl alloy variants on edge and near-edge computing devices. Then, we will show how object detection and tracking can be coupled with 2-tilt stereomicroscopy via the Obtain3D code package⁷ to form 3D reconstructions allowing for detailed 3D analysis such as pair distribution evolution of defects and 1D/3D diffusion of slow-moving defects under irradiation. The capabilities of this algorithm ensemble for high fidelity quantification are realized through a platform that couples a modern web application (webapp) hosted on edge or near-edge computing devices that is seamlessly coupled with a state-of-the-art <i>in situ</i> TEM ion irradiation facility. This platform enables real-time application of the algorithm ensemble and corresponding graphical displays and overlays from the <i>in situ</i> video feed during irradiation providing for <i>in operando</i> microscopy. The edge-device with a hosted webapp platform, termed Theiascope^TM, has been demonstrated for <i>in situ </i>TEM ion irradiations. It will be shown the platform is system agnostic (both in hardware and experiment) allowing for immediate feedback of materials evolution when exposed to extreme environments in other imaging systems such as scanning electron microscopy. We will conclude the discussion with recent innovations and thinking on extending the overall presented framework and advances to automated microscopy experimentation through integration with microscopy vendor software APIs.<br/><br/>1. Li, W., Field, K. G. & Morgan, D. <i>NPJ Comput Mater</i> <b>4</b>, 36 (2018).<br/>2. Jacobs, R. <i>et al.</i> <i>Scientific Reports 2023 13:1</i> <b>13</b>, 1–13 (2023).<br/>3. Shen, M. <i>et al.</i> <i>Comput Mater Sci</i> <b>197</b>, 110560 (2021).<br/>4. Jacobs, R. <i>et al.</i> <i>Cell Rep Phys Sci</i> (2022).<br/>5. Shen, M. <i>et al.</i> <i>Comput Mater Sci</i> <b>199</b>, 110576 (2021).<br/>6. Field, K. G., Eftink, B. P., Saleh, T. A. & Maloy, S. A. https://doi.org/10.2172/1439930 (2018) .<br/>7. Eftink, B. P. & Maloy, S. A. obtain3D. https://www.osti.gov/servlets/purl/1371737 (2017).<br/>8. Field, K. G. <i>et al.</i> <i>Microscopy and Microanalysis</i> <b>27</b>, 2136–2137 (2021).<br/>9. Wadsworth, J. <i>et al.</i> <i>Basic Research Needs for Materials Under Extreme Environments, June 11-13, 2007</i>. (2008) doi:10.2172/935440.