Machine Learning Analysis and Automation in High-Resolution Scanning Probe Microscopy

Scanning Probe Microscopy (SPM) has been the engine of characterization in nanoscale systems in general, and the evolution of functionalized tips as a reliable tool for high-resolution imaging without material restrictions has been a breakthrough in studies of molecular systems. In parallel, machine learning (ML) methods are increasingly being applied to data challenges in SPM. In particular, the success of deep learning in image recognition tasks has led to their application to the analysis of SPM images, especially in the context of surface feature characterisation and techniques for autonomously-driven SPM [1].<br/><br/>In this work, we explore the potential for using ML approaches to aid in the analysis of high resolution Atomic Force Microscopy (AFM) and Scanning Tunnelling Microscopy images. Building upon a deep learning infrastructure that matches a set of AFM images with a unique descriptor characterizing the molecular configuration [2-4], we further develop a workflow that takes experimental images of complex molecular systems and revises initial ML structure predictions with neural network potential simulations. In this context, we discuss the challenges of handling experimental data and possible data augmentation strategies. Alongside this, we show how ML approaches can be used actively during SPM experiments to aid in both tip functionalization [5] and in the construction of nanostructures through atomic manipulation [6], while also highlighting approaches towards automated construction of complex systems.<br/><br/>[1] O.M. Gordon and P.J. Moriarty, Mach. Learn.: Sci. Technol. 1 (2020) 023001<br/>[2] B. Alldritt, P. Hapala, N. Oinonen, F. Urtev, O. Krejci, F. F. Canova, J. Kannala, F. Schulz, P. Liljeroth, and A. S. Foster, Sci. Adv. 6 (2020) eaay6913<br/>[3] Niko Oinonen, Chen Xu, Benjamin Alldritt, Filippo Canova, Fedor Urtev, Ondrej Krejci, Juho Kannala, Peter Liljeroth and Adam S. Foster, ACS Nano 16 (2022) 89<br/>[4] Niko Oinonen, Lauri Kurki, Alexander Ilin and Adam S. Foster, MRS Bulletin 47 (2022)<br/>[5] Benjamin Alldritt, Fedor Urtev, Niko Oinonen, Markus Aapro, Juho Kannala, Peter Liljeroth and Adam S. Foster, Comp. Phys. Comm. 273 (2022) 108258<br/>[6] I-Ju Chen, Markus Aapro, Abraham Kipnis, Alexander Ilin, Peter Liljeroth and Adam S. Foster, Nat. Commun. 13 (2022) 7499