Scientists have adapted a ChatGPT-like system to answer questions about chemistry research.Credit: Luis Alvarez/Getty
With only a little fine-tuning, a machine-learning system similar to ChatGPT can become surprisingly adept at answering research questions in chemistry. When predicting the properties of molecules and materials or the yields of reactions, the general-purpose system can match or beat the performance of more specialized models while requiring a smaller amount of tweaking, researchers write today in Nature Machine Intelligence1.
These results suggest that chatbots trained in a similar way to ChatGPT could become powerful tools for chemistry laboratories that don’t have the resources to develop or purchase sophisticated machine-learning models. “This greatly reduces the barrier for other chemists to benefit from machine learning in their domains,” says Andrew White, a chemical engineer at the University of Rochester in New York.
Chemical training
Large language models (LLMs) are artificial neural networks trained on huge collections of text. When prompted with a statement or a question, the systems can generate a response by statistically predicting how to follow one sentence with the next.
Computational chemist Kevin Jablonka, now at the Friedrich Schiller University of Jena in Germany, and his collaborators wanted to see what general-purpose LLMs could do for chemistry. They started with GPT-3, an early iteration of the ‘brain’ behind the ChatGPT chatbot that became a global sensation after OpenAI in San Francisco, California, launched it in late 2022.
This GPT-powered robot chemist designs reactions and makes drugs — on its own
To adapt GPT-3 to answer questions about a chemical compound or material, the researchers first collected information from the literature about similar compounds or materials, and formatted their data to take the form of up to 30 questions and answers. They then sent the data to OpenAI to be added to the LLM’s training set. The fine-tuned system could answer predictive questions about the original compound or material — even though it was not explicitly included in the input data. “What’s remarkable is that it can do things it doesn’t know,” says co-author Berend Smit, a chemical engineer at the Swiss Federal Institute of Technology in Lausanne.
For example, the researchers tested the system’s aptitude for answering queries about ‘high entropy’ alloys, which are made of roughly equal amounts of two or more metals. Ordinary alloys such as steel — which contains mostly iron, with small amounts of elements such as carbon mixed throughout its crystal structure — are well understood, but much less is known about high-entropy alloys and how their metals will mix. However, the fine-tuned LLM could guess correctly how the metals in one of these alloys would arrange themselves. (The researchers assessed success by leaving some alloys from the literature out of the training data and then testing whether the LLM could predict those alloys’ properties.)
An LLM for the people
When asking the system to answer questions about an ‘unknown’ material not included in the training data, the team got results comparable in accuracy to those of more specialized machine-learning tools for chemistry, and even to those from computer simulations explicitly programmed to use the physical properties of atoms and molecules. The researchers also demonstrated that they could achieve similar results when they fine-tuned an open-source version of GPT-3, called GPT-J — meaning that labs with small budgets might be able to develop their own version without having to pay or ask for commercial help.
ChatGPT has entered the classroom: how LLMs could transform education
“The ‘democratization’ is perhaps one of the more interesting things about this project, as it indeed makes it way easier to get some predictions of chemical properties,” Jablonka says.
The fact that the technique can get predictions just from the chemical formula for a compound is “very surprising”, White says. He tried the method for himself as soon as he saw Jablonka and colleagues post their findings on the preprint server ChemRxiv a year ago, ahead of peer review. “We have used it in new projects — like designing new catalysts based on fine-tuning LLMs,” says White. “It is the first method we try when working on new projects.”
Although the method requires humans to collect information and prepare the LLM input, Jablonka and his team aim to design future versions that can implement this step automatically by mining text from existing literature.