Matbench Discovery 2.0: Benchmarking Machine Learning Force Fields on (an)Harmonic Phonon Modeling

<b>Matbench Discovery</b> (https://matbench-discovery.materialsproject.org) has attracted academic and industry interest for benchmarking ML models on high throughput crystal stability prediction (250k materials). However, with the tremendous advance of ML potentials, the leaderboard has quickly saturated and shown itself prone to overfitting on the MPtrj training set. <br/>This talk will cover ongoing work to pivot MatBench Discovery away from energy-only metrics towards harmonic and anharmonic phonon modeling incl. imaginary mode classification and thermal conductivity prediction.<br/>As these new metrics are based on 2nd and 3rd order potential energy surface (PES) derivatives, we believe they make for much stricter tests of how smooth/physical the PES learned by different MLFFs are. Many of the investigated properties are also experimentally accessible, opening up pathways for measuring the benefit of fine-tuning foundational MLFFs on experimental data for potentially exceeding DFT accuracy in property predictions.