Amir Barati Farimani: Learning the Language of Atoms: Generative Pretraining Transformers (GPT) for Materials and Molecules

Pretraining as a Foundation for Materials Innovation

Recent breakthroughs in large language models (LLMs) demonstrate their power to capture patterns across vast domains of knowledge. Materials science, with its immense design space and sparse experimental data, is uniquely positioned to benefit from this paradigm. In this talk, I will discuss how pretraining strategies—long successful in language and biology—can be reimagined for materials discovery.

By embedding atomic structures, chemical compositions, and simulation trajectories into transformer-based models, we can create powerful universal representations of matter. These pretrained models accelerate downstream tasks such as property prediction, generative design of novel compounds, and multi-scale simulation.

I will also highlight opportunities for integrating LLMs with physics-based priors and high-throughput experimentation to create closed-loop, autonomous material discovery pipelines. Ultimately, I will argue that pretraining is not just a technical trick, but a new foundation for materials innovation—one that can dramatically shorten the path from fundamental discovery to real-world deployment.

Our Speaker:

Amir Barati Farimani

Professor Amir Barati Farimani is the Russel V. Trader Associate Professor of Mechanical Engineering at Carnegie Mellon University, where he leads the Mechanical and Artificial Intelligence Lab (MAIL). His research integrates machine learning, molecular dynamics, and physics-based modeling to accelerate discovery in materials science, biology, and engineering.

He received his Ph.D. in Mechanical Science and Engineering from the University of Illinois at Urbana-Champaign in 2015 and completed a postdoctoral fellowship at Stanford University in computational biology and AI under Professor Vijay Pande. At CMU, his group develops foundation models and neural operators for molecular simulations, generative design of molecules and materials, and AI agents for scientific discovery.

He has published extensively on data-driven approaches to physics and chemistry, and his work has been supported by DARPA, ARL, NIH, and industry partners. His long-term vision is to create autonomous, AI-driven pipelines that can rapidly design, simulate, and experimentally validate new materials and molecules.