Sandia National Labs Visualizes Some Of The World's Largest Simulations Using NVIDIA® Technology

National Laboratories Leverage NVIDIA Quadro® Graphics and ParaView Software to Render Extremely Large Datasets in Record Time


David Higham     
NVIDIA Corporation
(408) 486-8287

Brian Wylie
Sandia National Labs
(505) 844-2238

Neal Singer
Sandia National Labs
(505) 845-7078


ALBUQUERQUE, NM, CLIFTON PARK, NY, and SANTA CLARA, CA–November 15, 2005–Sandia National Labs, Kitware, Inc., and NVIDIA Corporation (Nasdaq: NVDA) today announced a major advance in large-data scientific visualization and rendering for simulations being run on Sandia's new supercomputer, Red Storm. Sandia's Red RoSE visualization cluster is being used to render this data, allowing scientists to analyze real world problems with extraordinary precision and fidelity in fields such as national defense and security, aerospace and environmental research.

These results were achieved with ParaView (www.paraview.org), an open source visualization application, running on Red RoSE, one of the world's most powerful visualization clusters.  Developed by Kitware, Inc. (www.kitware.com), ParaView enables parallel graphics processing across a large number of NVIDIA Quadro cards in the Red RoSE visualization cluster.  This achievement stems directly from sustained efforts to deliver scalable visualization technologies through strategic investments and developments by the National Nuclear Security Administration's (NNSA) Advanced Simulation and Computing (ASC) program and the ASC tri-labs, Sandia National Laboratories (SNL), Los Alamos National Laboratory (LANL), and Lawrence Livermore National Laboratory (LLNL).

Visualization is an integral component of ASC and is essential to understanding the massive data produced in simulations being run in the national labs. Such simulation results can be enormous (often terabytes in size) and can provide a unique challenge to the visualization systems tasked with the processing of this data.
The "Red RoSE" cluster, which is part of the infrastructure deployed at Sandia in support of the Red Storm environment, includes 264 visualization nodes (workstations), each comprised of the following:
-- NVIDIA Quadro FX 3400 PCI Express graphics boards
-- Workstations equipped with Dual 3.6 GHz Intel Xeon EM64T processors, and 4GB of RAM
-- InfiniBand 4x HCA interconnect, allowing the data to be processed across the 264 nodes

"The combination of ParaView and NVIDIA high-performance graphics hardware has greatly enhanced our scientific visualization abilities for large data sets. We're now able to load up, process, and visualize some of the world's largest datasets at interactive rates," said Brian Wylie, visualization team leader, Sandia National Labs.

Caption: Asteroid Golevka measures about 500 x 600 x 700 meters. In this CTH shock physics simulation, a 10 Megaton explosion was initiated at the center of mass. The simulation ran for about 15 hours on 7200 nodes of Red Storm and provided approximately 0.65 second of simulated time. The resolution was 1 meter, with a 1 cubic kilometer mesh that contained 1.1 billion cells. The remarkable resolution of this simulation provides realism in crack formation and propagation not seen in lower-resolution models.

"NVIDIA Quadro commercial off-the-shelf solutions provide the graphics processing power in visualization systems that have traditionally relied on costly proprietary supercomputers," said Jeff Brown, general manager for NVIDIA professional products, "NVIDIA professional platforms allow our customers like Sandia to deploy solutions that are highly scalable and take advantage of the very latest advances in GPU technologies."

ParaView performs various operations on the data including calculating derivative scalar fields, t-stripping, cutting, clipping, and glyphing at interactive rates. The aggregate parallel rendering performance is delivered via a high performance image delivery system to the user's desktop. For typical simulation results, ParaView streams images from a cluster to a desktop at about 15 frames per second. This level of performance is enabled by the latest generation of NVIDIA Quadro graphics hardware which provides fast PCI Express read-back rates. ParaView can deliver rendering performance of over 8 billion polygons per second through Sandia's parallel rendering library (Ice-T) and Sandia's image compression system (SQUIRT).

ParaView is built on top of the popular Visualization Toolkit (VTK) and leverages cutting-edge parallel rendering algorithms and leading technology in commodity PC clusters and graphics hardware to interactively visualize some of the world's largest datasets. Contributors and developers currently include: Kitware, LANL, SNL, and the Army Research Laboratory. ParaView is funded by the US Department of Energy ASC program as part of a contract awarded to Kitware, Inc. by a consortium of the three ASC National Labs - LANL, SNL, and LLNL. The goal of the project is to develop scalable parallel processing tools with an emphasis on distributed memory implementations. The project includes parallel algorithms, infrastructure, I/O, support, and display devices. One significant feature of the contract is that all software developed is to be delivered open source. Hence ParaView is available as an open-source system.

About Sandia
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin company, for the U.S. Department of Energy's National Nuclear Security Administration. Sandia has major R&D responsibilities in national security, energy and environmental technologies, and economic competitiveness.

About Kitware
Kitware Inc. is a small business with expertise in imaging, 3D graphics, visualization and object-oriented software design. Founded in 1998, Kitware has grown to include customers in over 43 countries. Kitware's primary customers include the oil & gas industry, medical imaging, government research, computer aided design & manufacturing, aerospace, and engineering. Kitware's mission is to provide state of the art visualization, graphics, and image processing software solutions. This includes developing turn-key, end user applications; creating customized applications for clients; porting our open-source tools such as VTK, ITK, and CMake to custom hardware; supporting these open-source software tools with documentation and tools; providing professional consulting services.

NVIDIA Corporation is the worldwide leader in programmable graphics processor technologies. The Company creates innovative, industry-changing products for computing, consumer electronics, and mobile devices. NVIDIA is headquartered in Santa Clara, CA and has offices throughout Asia, Europe, and the Americas. For more information, visit www.nvidia.com.

Certain statements in this press release including, but not limited to, the benefits, features, performance, and capabilities of NVIDIA products and NVIDIA technology, including NVIDIA professional platforms and uses of Red Storm are forward-looking statements that are subject to risks and uncertainties that could cause results to be materially different than expectations.  Such risks and uncertainties include, but are not limited to, manufacturing or software defects or bugs, our dependence on third party manufacturers,  the impact of new products or technologies, the impact of competitive products and pricing alternatives, changes in industry standards and interfaces, our dependence on third-party developers and publishers and other risks detailed from time to time in the NVIDIA reports filed with the Securities and Exchange Commission including its Form 10-Q for the quarter ended July 31, 2005. These forward-looking statements speak only as of the date hereof. NVIDIA disclaims any obligation to update these forward-looking statements.


Cray and Sandia have designed and developed a new massively parallel supercomputer, Red Storm, which contains over 10,000 AMD Opteron processors in a single system that is functionally and physically integrated. Sandia helped design a novel interconnect that enables high bandwith, low latency communications between the processors and also developed a highly efficient light-weight operating system for the compute nodes. Sandia is using the new supercomputer to solve many large-scale problems including nuclear weapons safety, climate and asteroid deflection.

Visualization nodes of Sandia's Red RoSE cluster.  Red Rose provides interactive visualization, analysis and management of the output from very large simulations.  The cluster has 264 visualization nodes, which are Dell Precision 470 workstations with dual Intel Xeon EM64T CPUs and  NVIDIA Quadro FX 3400 PCI Express graphics boards.  These workstations are interconnected with a Voltaire InfiniBand network to provide very high-bandwidth communications among the workstations and with the parallel filesystem that stores the output from the simulations. 

150 million degree-of-freedom volume-rendered loosely coupled SIERRA/Fuego/Syrinx/Calore object-in-cross-wind fire (one fluids and participating media radiation region; two conduction regions). The simulation was run on 2048 Red Storm processors. The simulation is part of a qualification test plan for system testing to be conducted at the new Thermal Test Complex Cross Wind Facility (XTF).

This terabyte simulation models the breakdown of the polar vortex, a circumpolar jet that traps polar air at high latitudes creating conditions favorable for ozone depletion. The breakdown of the vortex, which occurs once or twice a year in the polar wintertime stratosphere, can transport this ozone-depleted polar air well into the mid latitudes.


Copyright© 2013 NVIDIA Corporation. All rights reserved. All company and/or product names may be trade names, trademarks, and/or registered trademarks of the respective owners with which they are associated. Features, pricing, availability, and specifications are subject to change without notice.

Note to editors: If you are interested in viewing additional information on NVIDIA, please visit the NVIDIA Press Room at http://www.nvidia.com/page/press_room.html