A 3D molecule model.

Healthcare & Life Sciences

Enhancing Biologics Discovery and Development With Generative AI

Objective

Using NVIDIA DGX™ Cloud and BioNeMo™, Amgen trains large language models (LLMs) on their proprietary data to help predict properties of proteins and develop biologics with enhanced properties.

Customer

Amgen

Use Case

Generative AI / LLMs

Products

NVIDIA BioNeMo service
NVIDIA AI Enterprise
NVIDIA DGX

Training LLMs to Design and Predict Properties of Proteins, Speeding Up Drug Discovery

As one of the world's leading biotechnology companies, Amgen is known for being a pioneer in biologics. From drugs that combat severe arthritis, anemia, and other inflammatory diseases to treatments for cancer, Amgen has developed some of the largest-selling pharmaceutical products that have improved hundreds of thousands of lives.

One of Amgen's key R&D focus areas is biologics, which are complex molecules that are made in and extracted from living cells. The complexity of biologics allows them to be engineered to bind specifically to a disease-causing agent and reduce its effects. To accelerate the pace of drug discovery, Amgen sought to use artificial intelligence and machine learning to design these large and complex molecules.

Amgen headquarters.
Image courtesy of Amgen.

Summary

  • Traditional biologics discovery is very costly, involving target identification from tens of thousands of molecules, candidate selection from millions of molecules, and rigorous clinical testing.
  • To accelerate biologics discovery, Amgen is using generative AI models to propose designs for candidate molecules and predictive models to evaluate designs.
  • They leveraged NVIDIA DGX Cloud and NVIDIA BioNeMo for rapid training and fine-tuning of protein LLMs and NVIDIA RAPIDS for up to 100X faster post-training analysis.
  • BioNeMo on DGX Cloud is a turnkey solution that enabled Amgen to get up and running quickly, moving from initial login to training large models in just a few days.

Building and Maintaining Robust AI Infrastructure for Biomolecular LLMs

The traditional process of discovering new therapeutics involves four phases: target selection, where potential drug targets are identified; lead discovery and optimization, where potential therapeutics are identified and optimized; candidate selection, where molecules are chosen to be further developed; and clinical development, where the safety and effectiveness of the drug are tested. This process is long and costly: You could start from thousands to millions of unique antibodies or other proteins, select hundreds for high-throughput screening, and from there end up with a small set of lead molecules. Amgen wanted to develop AI and machine learning tools to accelerate screening and optimization. 

Large language models and generative AI can analyze data and predict outcomes, enabling Amgen researchers to develop new biologics with greater speed and accuracy. LLMs use data from vast protein sequence databases to create a virtual version of a biologic, which can then be used to generate hypotheses about the effects of the biologic, its properties, and its potential side effects. However, some subclasses of biologics are new to nature, especially multi-specific molecules, and data is sparse, so it can be challenging to make in silico predictions about them. "Because publicly available models are limited, we needed to pretrain custom models on our proprietary data," says Christopher Langmead, director of digital biologics discovery at Amgen. "Pretraining such models and then performing inference at scale requires powerful compute and a highly optimized software and hardware platform."

Solution

NVIDIA DGX Cloud

  • DGX Cloud instances, each with eight NVIDIA A100 80GB Tensor Core GPUs
  • NVIDIA Base Command™ Platform for job scheduling and orchestration
  • NVIDIA AI Enterprise, including RAPIDS for reading large inputs and clustering results

NVIDIA BioNeMo

  • Training and Inferencing state-of-the-art biomolecular models with a focus on proteins

Results

  • Faster training of protein LLMs compared to open-source options
  • Faster protein structure predictions—as fast as 20 seconds per structure
  • Less than four weeks from onboarding on DGX Cloud to the first pretrained protein LLM model

On-Demand Supercomputing Resources and Customizable Generative AI Models

Amgen developed a generative biology workflow using AI and machine learning that begins with a set of specifications a candidate must satisfy. Next, generative AI models suggest new designs, and predictive models evaluate and rank these designs. This is done iteratively until molecules are found that satisfy the specifications, which include criteria relevant to efficacy, safety, and manufacturability. Evaluating as many designs in silico with these generative models reduces the burden on wet labs.

“To develop models that can help us generate good biologics, we needed our platform to support rapid pretraining and fine-tuning across a range of experiments,” says Langmead. “We needed the flexibility to experiment with different data and scale. Using NVIDIA BioNeMo on DGX Cloud, we were able to easily perform distributed training of complex models in a multi-GPU environment. The capabilities and performance of NVIDIA BioNeMo and DGX Cloud were precisely what we needed and available to us when we needed them.”

“One of the key advantages of DGX Cloud was the remarkably swift onboarding process. We were able to progress from our initial login to pretraining large models in just a few days. BioNeMo on DGX Cloud is a turnkey solution—our users only need to supply data and specify the model by adjusting a few configuration files, and BioNeMo handles all other aspects of the process.”

Amgen trained protein LLM ESM-1nv in BioNeMo on DGX Cloud with Amgen proprietary antibodies. This resulted in five trained antibody-specific LLMs. BioNeMo has state-of-the art biomolecular large language and diffusion models for training and inferencing in early-stage drug discovery workflows. This includes models for generating proteins and small molecules, understanding protein and small molecule properties, predicting binding structures of small molecules bound to proteins, and predicting the 3D structure of proteins.

“Ease of multi-node training and the ability to use larger batch sizes within DGX Cloud enabled us to achieve our three-month objectives in just four weeks.”

Chris James Langmead,
Director of Digital Biologics Discovery, Amgen

Faster Training of Protein LLMs and Up to 100X Faster Post-Training Analysis

Langmead commented, "Ease of multi-node training and the ability to use larger batch sizes within DGX Cloud enabled us to achieve our three-month objectives in just four weeks. Multi-node and multi-GPU training are important in biologics, because they can help speed up the training process and enable the training of larger models with more data. This leads to more accurate models and predictions, which accelerate the drug development process.”

DGX Cloud is optimized for multi-node training, allowing Amgen to experience remarkable speedups. “We saw dramatic speedups due to distributed training and optimized data loaders using the DGX platform relative to a single-GPU environment.”

Using NVIDIA Base Command Platform within DGX Cloud, Amgen’s researchers could submit all jobs with ease. Monitoring and telemetry features ensured all jobs ran smoothly and efficiently. “Base Command Platform was very intuitive. This ability to align our compute resources without concerning ourselves about the intricacies of distributed training in a multi-GPU and multi-node environment permits my team to concentrate on the scientific work and deliver models and tools at a quicker pace than would have been feasible in any other setting,” says Langmead.

BioNeMo also includes an accelerated implementation of the OpenFold model, a biologic modeling technique that uses a physics-based approach to predict the 3D structure of proteins. Predicting 3D structures of proteins helps researchers gain insights into the protein's functionality and develop more effective and targeted biologics that can bind to the target protein and improve therapeutic outcomes. “Compared to Amgen's own internalized version of the same model, we saw 20–30X speedups for creating multiple sequence alignments in BioNeMo. Separately, using a publicly available model for protein structure prediction, we saw dramatic speedups,” Langmead says.

“The powerful computing and multi-node capabilities of DGX Cloud has enabled Amgen to achieve faster training of protein LLMs with BioNeMo and up to 100X faster post-training analysis with NVIDIA RAPIDS.”

Chris James Langmead,
Director of Digital Biologics Discovery, Amgen

"With NVIDIA DGX Cloud and NVIDIA BioNeMo, our researchers are able to focus on deeper biology instead of setting up AI infrastructure. The powerful computing and multi-node capabilities of DGX Cloud has enabled Amgen to achieve faster training of protein LLMs with BioNeMo and up to 100X faster post-training analysis with NVIDIA RAPIDS relative to alternative platforms.” 

“The responsiveness of the NVIDIA AI experts who are technical resources in getting our codes running efficiently on their platform was key. Instead of going to forums, we got answers about our infrastructure and tooling in real time. Because of this, my team can focus on modeling, not software engineering."

Looking Forward

Enhancing the discovery and development of biologics promises to deliver more effective treatments with improved manufacturability and reduced or eliminated side effects at lower cost. To achieve this, Amgen is seeking to expand workloads using BioNeMo on DGX Cloud. This includes pretraining application-specific protein language models and RNA language models and deploying these models on Amgen's generative biology platform. "DGX Cloud and BioNeMo provide the performance and scalability we need, enabling increased productivity and impact. It’s already transforming the way we work by dramatically increasing the scope of what can be accomplished by a team our size," says Langmead.

Ready to Get Started?

To learn more about NVIDIA solutions for healthcare and life sciences, contact us.