Search Trees in Large Continuous Action Spaces for Multiscale Modeling and Design of Materials

Modeling materials at various scales is of utmost importance in a diverse array of practical applications, allowing us to predict and gain insights into the favorable properties of novel materials, including super-hardness, electronic behaviors, and catalytic potential, among others. The understanding of material properties is intricately linked to the underlying structure and the potential energy landscape. Any approach to material modeling encompasses two key aspects: (a) the use of high-dimensional potential energy models to accurately predict material properties and (b) the navigation of the energy landscape to identify atomistic configurations that possess these desired traits. While first-principles-based approaches have demonstrated accurate property predictions, scalability remains a significant challenge, particularly in systems with heterogeneity and sizes that extend to scales with significant practical implications. Traditional potential energy models provide a cost-effective alternative, but their parameterization becomes challenging due to the dimensionality and continuous nature of search space. Additionally, the complexity of the energy landscape, exhibited by these potential models, makes it difficult to navigate toward local minima corresponding to metastable phases or locate global minima for inverse design problems. The high dimensionality of these problems necessitates the implementation of efficient approaches that can effectively explore the search space, capture the diversity of polymorphs, and converge toward desired properties. Drawing inspiration from the remarkable success of tree search algorithms in policy-driven reinforcement learning for games like Chess, Shogi, and Go, we have developed a continuous search space adaptation of Monte Carlo Tree Search to address these challenging material design problems. Our contribution to this continuous space adaptation includes modifying the rewards scheme to enhance exploration, implementing a "funneling" scheme for improved exploitation, and incorporating adaptive sampling during playouts to achieve efficient and scalable search. Using standardized high-dimensional artificial landscapes, we have successfully benchmarked our approach against popular metaheuristics-based global optimization techniques and state-of-the-art policy gradient methods. Our applications include exploring high-dimensional potential energy models of representative systems, such as nanoclusters of elements like Al, C, and Cu. Additionally, we have demonstrated the ability of our method to predict global minima crystal structures in systems with different dimensionalities and compositions, ranging from 0D (Au nanoclusters) to 2D (MoS2) and oxides like TiO2. Furthermore, we have extended our approach to addressing representative continuum-scale modeling problems, such as minimizing fabrication costs in welded joints or optimizing the weight of tension/compression springs. Additionally, we have applied our method to playing high-dimensional video games, showcasing its versatility. Future applications include implementing these algorithms to facilitate the discovery, exploration, and learning of synthesis pathways driven by protocols to enhance the understanding of phase transformations in material systems.