In Situ Observation of Crystallization Processes Using LC-TEM with the Support of Machine Learning

We have observed the crystallization processes of hen egg-white lysozyme protein [1] and the aggregation processes of amyloid-β protein [2], which is considered to cause Alzheimer's disease, by liquid-cell transmission electron microscopy (LC-TEM). In these observations, it is necessary to examine the issues of water radiolysis and electron damage to the sample due to electron irradiation. For example, in the crystallization experiment of lysozyme protein, the dependence of crystal growth rate on electron dose was observed. Therefore, we determined the threshold electron dose rate at which the crystal growth rate measured under an optical microscope was maintained, and made observations at lower electron dose rate than the threshold. Fortunately, the electron dose rate was not so small, 320 electron nm^-2 s^-1, and therefore, there were no significant problem with observation at low magnifications. However, for observation at a high magnification or when the sample is more sensitive to electron beams, it is necessary to lower the electron dose rate, resulting in a dark image for observation. Although post-processing can brighten the image and increase contrast, we must make decisions about observation and experimental conditions during in-situ observation based on the dark image. We have therefore developed an algorithm of machine learning to enable image sharpening through real-time image processing during in-situ observation [3].<br/><br/>Another challenge for in situ experiments of crystallization is the detection of particles; in a typical in situ TEM, it is difficult to save all images during the experiment. Therefore, using the look back function provided in the TEM camera, recording is started when the phenomenon is confirmed to have occurred, starting with the image from a few seconds earlier. However, because the human eye has difficulty noticing slow phenomena, it may not be possible to record the initial process at the time we notice it. Therefore, we developed a machine-learning algorithm to detect the initial stage of the phenomenon by real-time image processing. This has made it possible to detect nanoparticles generated from solution through nucleation as quickly as possible [4].<br/><br/>By modifying an algorithm for particle detection, we used machine learning to analyze the relationship between the nucleation of nanoparticles from solution and the dissolution process [5]. In aqueous solution, clustering of ions and degradation of clusters are taking place, and we hypothesized that this is recorded as fluctuations in the TEM images. Therefore, we attempted to predict the formation of nanoparticles by machine learning using images taken several images before the time when the particles were formed. By changing the time period for learning, it may be possible to determine how many seconds before the nanoparticles form, clustering is occurring. Also, if particles undergo the same structure as clustering when they dissolve, machine learning should falsely detect the formation of new particles at the moment they dissolve. I will introduce our latest results of in-situ observation of crystallization with a support of machine learning.<br/><br/>[1] T. Yamazaki, Y. Kimura, P. G. Vekilov, E. Furukawa, M. Shirai, H. Matsumoto, A. E. S. Van Driessche, K. Tsukamoto, <i>Proc. Natl. Acad. Sci. USA</i>, <b>114</b> (2017) 2154.<br/>[2] K. Nakajima, T. Yamazaki, Y. Kimura, M. So, Y. Goto, H. Ogi, <i>J. Phys. Chem. Lett.</i>, <b>11</b> (2020) 6176.<br/>[3] H. Katsuno, Y. Kimura, T. Yamazaki, I. Takigawa, <i>Microscopy and Microanalysis</i>, <b>28</b> (2022) 138-144.<br/>[4] H. Katsuno, Y. Kimura, T. Yamazaki, I. Takigawa, <i>Frontiers in Chemistry</i>, <b>10</b> (2022) 818230.<br/>[5] Y. Kimura, H. Katsuno, T. Yamazaki, <i>Faraday Discussions</i>, (2022) in press. DOI: 10.1039/D1FD00125F