CUDA Spotlight: GPU-Accelerated Molecular Dynamics

This week's spotlight is on Dr. Ross Walker, assistant research professor with the San Diego Supercomputer Center (SDSC) at UC San Diego and adjunct assistant professor in the Department of Chemistry and Biochemistry at UC San Diego.

Dr. Walker is a lead developer of AMBER (Assisted Model Building with Energy Refinement), a Molecular Dynamics (MD) software package for the simulation of biomolecules. A new update of AMBER, focusing on improved NVIDIA GPU acceleration, was recently made available to the public.

NVIDIA: Ross, tell us about the recent news announcement.
Ross: The announcement concerns a large scale update to the GPU acceleration support in AMBER. This is something we have been working on for the last nine months with funding from the National Science Foundation's Scientific Software Infrastructure and Innovation program, in close collaboration with NVIDIA.

The update more than doubles the performance we achieve on NVIDIA GPUs while adding significant new functionality such as extra points and new implicit solvent models. This release provides previously unheard of capabilities while maintaining AMBER's position as the most efficient and comprehensive GPU-accelerated Molecular Dynamics (MD) code.

NVIDIA: What are potential applications of GPU-accelerated AMBER?
Ross: The latest update to GPU-accelerated AMBER provides performance that was previously unattainable, even with the fastest supercomputers. Thus we are now in a realm where for the first time we are talking about increasing capability and not just GPU acceleration. This widens the scope for the types of scientific questions one can attempt to solve.

Simulations of drug binding using MD running on GPU-accelerated desk side systems are now becoming routine. For example, my own team, along with collaborators at UC Irvine, is using the code to uncover key activity crucial for drug discovery in H1N1 (swine flu) neuraminidase enzymes and also in the development of new treatments for Chlamydia. With over 10,000 scientists using the AMBER code for MD simulation of enzymes, the potential applications have broad applicability including drug discovery and optimization; biocatalysis applicable to biofuel development; and next generation materials development.

NVIDIA: How does GPU computing play a role in your work?
Ross: My work consists of three main thrusts. The first is in the underlying development of the key computational tools that others use for research while the second involves the development of more accurate force fields and mathematical models for looking at biochemical systems. The third involves applying the results of the first two to active research programs in drug discovery and improved biocatalysis. GPUs feature prominently in all three of these areas.

While my research group actively develops the GPU support for the AMBER software, we also use this in our research. For example we are developing, in collaboration with co-workers at the University of Bergen in Norway, new modular force fields for the simulation of lipid membranes, key to the accurate modeling of drug interactions with cell membranes and membrane bound proteins. The GPU-accelerated code is critical to being able to achieve the performance and throughput we need to accurately test and refine these force fields. At the same time, we are using GPUs to look at the atomistic mechanisms of drug interaction with influenza neuraminidases and adenovirus proteases (a key target for next generation antiviral drugs).

NVIDIA: What kind of advantages have you achieved with CUDA?
Ross: I cannot begin to do justice to the revolution that GPU acceleration through CUDA is having on Molecular Dynamics. The GPU acceleration of MD through CUDA is, for the first time, making it possible for scientists to 'experiment' computationally. Simulations are now fast enough, and the hardware easily accessible and affordable enough that scientists can actually obtain feedback in real time. Questions such as, what happens if I mutate this residue, tweak this force field parameter, etc. can now be asked and the answers obtained over a coffee break instead of weeks of batch scheduled computing.

With our latest update, the performance of the code on a single Tesla M2090 GPU outstrips that achievable on 192 nodes of a Cray XT5 supercomputer. Put four of these M2090 GPUsin a workstation under your desk and you are in the middle of a revolution in terms of what you can simulate and for how long.

Simulations of enzymes in explicit solvent for multiple microsecond long sampling times that previously required access to large scale national supercomputer resources for many months, can now be done routinely on a deskside system in a few weeks. This is fundamentally changing the approach used in computer simulation of enzymes. With continued long term funding from the National Science Foundation (NSF) and collaboration with NVIDIA, this work has the potential to fundamentally change the field of molecular dynamics.

NVIDIA: Why did you decide to work with the Keeneland Project?
Ross: The Keeneland project represents NSF's first large scale deployment of GPUs in a machine available to all US scientists. Given that AMBER is the most mature GPU-accelerated MD code available, it made sense for us to work closely with the Keeneland developers to ensure that it was optimized for the machine and available to all users. With these latest updates a single node of Keeneland provides performance exceeding that achievable on all of the previous NSF funded supercomputer resources. At the same time, the sheer scale of Keeneland is exciting because of the possibilities for more novel computational approaches that it presents.

For instance, we are working hard to support things such as 2D replica exchange that allows one to ask what happens to an enzyme drug complex as we change multiple environmental conditions. We could, for example, generate a profile providing changes with respect to temperature in one dimension and pH (or acidity) in the other. Doing these simultaneously lets us look for subtle correlations between these two variables. This has the potential to scale to thousands of GPUs and provide data on a finer scale than we have previously considered possible.

I am excited by the Keeneland project and other developments for GPU-accelerated supercomputers. Through our continued NSF funding and close collaboration with NVIDIA we will be ideally placed to enable both my own group and all users of the AMBER software to make full use of the full production Keeneland system from the first day of its deployment.

NVIDIA: How did you become interested in Molecular Dynamics?
Ross: My Ph.D. was based on developing models for understanding and interpreting results from femtosecond spectroscopy experiments. My co-researchers had developed an experimental technique for effectively carrying out nuclear magnetic resonance (NMR) with light. This allows one to actually 'photograph' chemical reactions as they occur but the interpretation of the results is non-trivial. My work was on developing a theoretical framework to interpret the experimental results and a key component of this was being able to simulate the motion of enzymes on the same timescales as our light pulses. In particular, I wanted to look at how an enzyme would relax from being photonically excited at femtosecond resolution.

Molecular Dynamics was the only method available that could be adapted to this purpose and so I very quickly got my hands dirty modifying popular MD codes for my purposes. This work exposed me to the possible applications of MD and in particular spiked my interest in what could be achieved if we could make microsecond or longer simulations feasible on readily available deskside systems. Following my Ph.D., I joined the lab of Prof. David Case at The Scripps Research Institute and from there became one of the major developers of the AMBER MD software package.

NVIDIA: When you think about the future of Molecular Dynamics, what excites you the most?
Ross: We are only just beginning to scratch the surface in the types of questions we can ask with Molecular Dynamics. Traditionally it has been too slow and computationally expensive to make a significant impact in the drug discovery process but with GPU acceleration we are now at a point where the accuracy of MD approaches is no longer limited by the amount of sampling that can be done.

For example, one could start using MD to do careful direct drug optimization. Consider having a drug bound in the active site of an enzyme. One could use MD-based thermodynamic techniques to calculate the differences in free energy of binding for a whole range of different mutations to the current known drug. This is something that previously would have been far too computationally expensive to be practical but now such techniques could become routine.

The sheer scope of research problems that were previously out of reach (due to their computational cost and the inability of MD algorithms to scale to hundreds of thousands of traditional CPU cores for systems of practical interest) means the field is now ripe for significant scientific discovery and impact.

Ross C. Walker is an assistant research professor with the San Diego Supercomputer Center (SDSC) and an adjunct assistant professor in the department of chemistry and biochemistry at UC San Diego where he runs the Walker Molecular Dynamics Lab.

Dr. Walker attended Imperial College of Science, Technology and Medicine, in London, UK, first as an undergraduate in Chemistry from 1996-2000 and then as a graduate student receiving his PhD in 2003 after completing his thesis on "The Development of a QM/MM Based Linear Response Method and its Application to Proteins."

He was named a CUDA fellow in September 2010 for his work accelerating molecular dynamics simulations on GPUs and he recently was named a recipient of the 2011 Outstanding Junior Faculty Awards presented by HP and the American Chemical Society's division of Computers in Chemistry (COMP).

Relevant Links:

Contact Info:
San Diego Supercomputer Center MC0505
University of California, San Diego
9500 Gilman Drive, La Jolla, CA, 92093-0505, USA
Tel: +1-868-822-0854