Facilitating In Situ 4D STEM with Advances in Software for Ultrafast Direct Detectors

Four-dimensional scanning transmission electron microscopy (4D STEM) involves imaging a diffracted electron beam with a pixelated detector at each probe position of a STEM scan. The resulting information-rich datasets allow multiple types of analyses of a specimen, including high-resolution structural visualization, and mapping of crystal grain structure, molecular orientation, strain, electric and magnetic fields, and more [1]. Typically, 4D STEM involves recording large (several Gigabyte to several Terabyte) 4D datasets to disk to be analyzed with post-acquisition software such as LiberTEM [2], py4DSTEM [3], or custom written code. Post-acquisition analysis enables a range of complex data analysis techniques to be applied to the data, but the efficiency of 4D STEM experiments can be severely impacted by the substantial time needed to go from data acquisition to data visualization.<br/><br/>Lack of real-time visualization poses challenges for both manual data acquisition, and for the development of automated 4D STEM workflows. However, with ultrafast direct detectors, such as the Celeritas XS camera, able to acquire data at rates approaching 5 Gigabytes per second, developing real-time 4D STEM analysis software involves its own significant challenges. These challenges are compounded when considering <i>in situ</i> 4D STEM, (sometimes referred to as 5D STEM), where the goal is not only to record one large 4D dataset, but multiple large 4D datasets over a period of time so that dynamic processes such as changes in strain, crystal structure, or electric fields in a specimen can be observed [4].<br/><br/>Here, we present a new 4D STEM software platform that aims to address these challenges by leveraging GPU processing for real-time virtual STEM image generation during repeated 4D STEM acquisition. The software is compatible with Direct Electron’s Celeritas XS, Celeritas, DE-16, DirectView2 and DE-64 cameras, as well as the DE-FreeScan scan generator, and allows the user to stream a series of full 4D STEM datasets to disk, even whilst simultaneously generating a live-view of up to four different types of virtual STEM image. In addition to viewing virtual images and camera frames in the software’s native interface, the system can be controlled via an API, allowing for integration into 3^rd-party software, as well as custom software for microscope automation.<br/><br/>Further developments are ongoing to augment this software platform with additional features with the goal of making 4D STEM and <i>in situ</i> 4D STEM more accessible techniques for electron microscopists to perform.<br/><br/>References:<br/>[1] Ophus C. (2019). <i>Microscopy and Microanalysis</i>, <b>25</b>, 563-582.<br/>[2] Clausen A. et al. (2020). <i>Journal of Open Source Software</i>, <b>5</b>, 2006.<br/>[3] Savitzky B. et al. (2021) <i>Microscopy and Microanalysis</i>, <b>27</b>, 712-743.<br/>[4] Huang, S. & Voyles, P. (2023) <i>Microscopy and Microanalysis</i>, <b>29</b>, S1, 272–273.