Symphony: Symmetry-Equivariant Point-Centered Spherical Harmonics for Generating Molecules

In-silico generation of diverse molecular structures has emerged as a promising method to navigate the complex chemical landscape, with direct applications to inverse material design and drug discovery. However, 3D molecular structure generation comes with several unique challenges; generated structures must be invariant under rotations and translations in 3D space, and must satisfy basic chemical bonding rules. Inspired by the success of machine learning models for generating coherent text and audio, there is much interest in building similar ‘generative models’ for 3D molecular structures.<br/>One of the first successful attempts was G-SchNet, an autoregressive model that uses message-passing with rotationally invariant features to generate 3D structures of small molecules. At each iteration, a focus node is selected as the center of a 3D grid, then all pre-existing atoms collectively decide on the next atom’s position within this grid. In this manner, G-SchNet generates molecules one atom at a time, completing molecular fragments into entire molecules. This construction raises several interesting questions; first, can we more accurately capture complex geometric motifs, and second, can we simplify the process of placing each atom without breaking symmetry via auxiliary tokens?<br/>Here, we present our work on Symphony, an E(3)-equivariant autoregressive generative model. E(3)-equivariant neural networks that utilize higher-order rotationally-equivariant features have recently shown improved performance on a wide range of atomistic tasks. Motivated by these results, Symphony builds on G-SchNet by using message-passing with higher-order equivariant features. This allows a novel representation of probability distributions via spherical harmonic signals.<br/>We will discuss the state of molecular generation models today and where Symphony sits amongst this rapidly changing landscape. We highlight several challenges we observed when developing Symphony, closely related to error accumulation and out-of-distribution generalization issues of machine learning methods. We present several applications of Symphony in generating small molecules and transition metal complexes, identifying key components of our model as well as areas of improvement. In particular, we demonstrate how higher-order equivariant features can model complex geometrical motifs which were unable to be captured by preceding methods such as G-SchNet. Finally, we discuss how hierarchical generation and building richer datasets beyond QM9 are important avenues to benchmark and improve generative models for molecules.