CUDA Spotlight: GPU-Accelerated Astronomy
This week's spotlight is on Dr. Denis Bastieri of the University of Padua, Italy and co-founder of Mimesis HPC. Dr. Bastieri leads the Fermi Large Area Telescope (LAT) team for the National Institute of Nuclear Physics (INFN) in Padua.
NVIDIA: Denis, tell us about the Fermi Space Telescope.
Denis: The Fermi mission is part of NASA's focus on the theme of "Structure and Evolution of the Universe.” I specifically work with one of the two instruments aboard the Fermi spacecraft -- the Large Area Telescope (LAT), which observes gamma rays, the electromagnetic radiation with the highest energy.
The subjects of the Fermi mission are no ordinary stars. The closest star to Earth, the Sun, cannot emit gamma rays at high enough energies to be detected by the LAT (unless a bright solar flare is occurring). What we are observing are collapsed stars like black holes and pulsars, stars as heavy as our Sun but with a typical size of only a few kilometers (smaller than the SF Golden Gate Bridge!), spinning around their axes many times per second, as well as far away galaxies and some Gamma Ray Bursts (GRBs), the most energetic explosions (second only to the Big Bang).
Fermi In Space
NVIDIA: Which organizations are involved in the project?
Denis: Fermi is a joint project between NASA, the U.S. Department of Energy and academic and research institutions across France, Germany, Japan, Italy and Sweden. The spacecraft was built by General Dynamics. Institutions in the LAT collaboration are listed at http://www-glast.stanford.edu/cgi-bin/collab_inst.
NVIDIA: What are some significant discoveries/findings?
Denis: Every time we look at our universe under a different wavelength we discover new unexpected phenomena. With the launch of Fermi from Cape Canaveral in June 2008, a new window on the gamma ray sky was opened and we expect a huge harvest of results.
Results to date include discovery of a new population of "gamma loud" pulsars previously undetected in radio; the observation of a record explosion from GRB 080916C in the constellation of Carina; and, in the constellation of Cygnus, we detected for the first time gamma rays that originated in a nova explosion.
Because gamma rays have the highest energies, they can scan deeper and deeper into the structure of our universe, so Fermi LAT is probing the nature of space-time itself. The Fermi mission is providing measurements that are currently under interpretation and will create the foundation for modeling the behavior of sources of high energies for many years to come.
The most important results may be seen on the NASA homepage of the Fermi project at: http://fermi.gsfc.nasa.gov/ or can be followed by downloading "Fermi Sky" for iPhone/iPad from the Apple App Store.
NVIDIA: What role does NVIDIA technology play?
Denis: Parallel computing is the only viable solution when dealing with many different aspects of astrophysics, and GPUs perform parallel computing at a tenth of the cost of conventional systems. Proof of the momentum behind GPUs can be seen in the exponentially growing number of astrophysics papers with "GPU" listed in the abstract!
In addition to modeling the behavior of fields and particles in the proximity of a supernova remnants, we have started to use NVIDIA technology (appropriately based on the Fermi architecture) to analyze data collected by the NASA Fermi satellite. Our analysis is not standard image processing; rather, we deal with likelihood computation. We sketch a model of a given sky region and we evaluate how well the model fits the data. Previously, the LAT collaboration had developed a few tools for this task, but analyzing data collected over one year or more was extremely time-consuming.
Today we are porting the lengthiest tasks to work under the NVIDIA Fermi architecture and by a careful refactoring of the code, optimizing for speed, we have seen a typical acceleration of about 100X. Together with SLAC, we are aiming at refactoring the data reduction, where raw data coming from the satellite (around 120 GB/day) is reduced into meaningful physical information, such as the direction and energy of detected gamma rays.
NVIDIA: What are the benefits of working with CUDA?
Denis: We started looking at parallel computing on GPUs back during the time of Cg (C for Graphics). The subject looked tantalizing: different textures for positions and velocities and we could model the evolution of a particle population in an external field. The results were quite promising, but Cg was not ideal for astrophysical modeling and we were almost going to dismiss the project entirely.
Then, CUDA was released in late 2006. We found programming in CUDA to be quite straightforward for any good C programmer. Our students demonstrated that they could become independent within a semester.
And now, CUDA 4.0 is even better! We utilize the Thrust algorithms library. We leverage Unified Virtual Address (UVA) to extend GPU memory capabilities. The CURAND library replaced our own version of a random number generator. CUDA allows us to fully exploit the performance of our 16 GPU cluster.
NVIDIA: What's next in the field of astronomy? What are you most excited about?
Denis: We are gaining more and more confidence in our modeling of the high-energy gamma ray emission of the universe. We understand by now what is the contribution from standard sources: pulsars, known galaxies, etc. But we are not able to explain all the data we collected! Something is missing and we have to understand what. Hints of a new population of gamma-ray emitters? The elusive Dark Matter starting to show up? Quoting Sherlock Holmes: "Whatever remains, however improbable, must be the truth!"
But this is just the "near" future: developing new, fast techniques to analyze data and find the improbable. The real future? We still have to shape it! We want to expand our GPU cluster, hire people, start new projects; in a word, we want to understand what it takes to build next-generation observatories. For a taste of what’s to come, keep an eye on the DARPA/NVIDIA exascale computing project!
Denis Bastieri learned computer programming in 1984, while on an exchange student program in Pittsburgh, PA. Since then he has developed control software for many different astroparticle physics detectors and, in parallel, videogames. He holds a PhD in Physics (astroparticle) and teaches geometrical and physical optics in the "Laboratory of Physics" course at the University of Padova. He co-founded Mimesis HPC together with Giorgio Urso, a fellow physicist with whom he shares a passion for parallel computing. Together they develop tools for simulation and data analysis. In his spare time he loves reading books, playing with his son and swimming in the sea.