NVIDIA HGX AI Supercomputer

The world’s leading AI computing platform.

Purpose-Built for AI and HPC

AI, complex simulations, and massive datasets require multiple GPUs with extremely fast interconnections and a fully accelerated software stack. The NVIDIA HGX™ AI supercomputing platform brings together the full power of NVIDIA GPUs, NVLink®, NVIDIA networking, and fully optimized AI and high-performance computing (HPC) software stacks to provide the highest application performance and drive the fastest time to insights. 

Read NVIDIA HGX H100 Datasheet
Read NVIDIA HGX A100 80GB Datasheet

Unmatched End-to-End Accelerated Computing Platform

The NVIDIA HGX H200 combines H200 Tensor Core GPUs with high-speed interconnects to form the world’s most powerful servers. Configurations of up to eight GPUs deliver unprecedented acceleration, with up to 1.1 terabytes (TB) of GPU memory and 38 terabytes per second (TB/s) of aggregate memory bandwidth. This combined with a staggering 32 petaFLOPS of performance creates the world’s most powerful accelerated scale-up server platform for AI and HPC.

Both the HGX H200 and HGX H100 include advanced networking options—at speeds up to 400 gigabits per second (Gb/s)—utilizing NVIDIA Quantum-2 InfiniBand and Spectrum™-X Ethernet for the highest AI performance. HGX H200 and HGX H100 also include NVIDIA® BlueField®-3 data processing units (DPUs) to enable cloud networking, composable storage, zero-trust security, and GPU compute elasticity in hyperscale AI clouds.

HGX Stack

Deep Learning Training: Performance and Scalability

Up to 5X Faster Training at Scale

Up to 5X Faster Training at Scale

Projected performance, subject to change.

GPT-3 175B training NVIDIA A100 Tensor Core GPU cluster: NVIDIA Quantum InfiniBand network, H100 cluster: NVIDIA Quantum-2 InfiniBand network

NVIDIA H200 and H100 GPUs feature the Transformer Engine, with FP8 precision, that provides up to 5X faster training over the previous GPU generation for large language models. The combination of fourth-generation NVLink—which offers 900GB/s of GPU-to-GPU interconnect—PCIe Gen5, and Magnum IO™ software delivers efficient scalability, from small enterprises to massive unified GPU clusters. These infrastructure advances, working in tandem with the NVIDIA AI Enterprise software suite, make HGX H200 and HGX H100 the world’s leading AI computing platform.

Deep Learning Inference: Performance and Versatility

Up to 30X Higher AI Inference Performance on the Largest Models

Megatron chatbot inference with 530 billion parameters.

Real-Time Deep Learning Inference

Inference on Megatron 530B parameter model chatbot for input sequence length = 128, output sequence length = 20, A100 cluster: NVIDIA Quantum InfiniBand network; H100 cluster: NVIDIA Quantum-2 InfiniBand network for 2x HGX H100 configurations; 4x HGX A100 vs. 2x HGX H100 for 1 and 1.5 sec; 2x HGX A100 vs. 1x HGX H100 for 2 sec.

AI solves a wide array of business challenges using an equally wide array of neural networks. A great AI inference accelerator has to not only deliver the highest performance, but also the versatility needed to accelerate these networks in any location—from data center to edge—that customers choose to deploy them. 

HGX H200 and HGX H100 further extend NVIDIA’s market-leading inference leadership, accelerating inference by up to 30X over the prior generation on Megatron 530 billion parameter chatbots.

HPC Performance

Up to 110X Higher Performance for HPC Applications

110x Higher Performance for HPC Adipplications

Projected performance, subject to change.
HPC MILC- dataset NERSC Apex Medium | HGX H200 4-GPU | dual Sapphire Rapids 8480
HPC Apps- CP2K: dataset H2O-32-RI-dRPA-96points | GROMACS: dataset STMV | ICON: dataset r2b5 | MILC: dataset NERSC Apex Medium | Chroma: dataset HMC Medium | Quantum Espresso: dataset AUSURF112 | 1x H100 | 1x H200.

Memory bandwidth is crucial for high-performance computing applications as it enables faster data transfer, reducing complex processing bottlenecks. For memory-intensive HPC applications like simulations, scientific research, and artificial intelligence, H200’s higher memory bandwidth ensures that data can be accessed and manipulated efficiently, leading up to 110X faster time to results compared to CPUs.

Accelerating HGX With NVIDIA Networking

The data center is the new unit of computing, and networking plays an integral role in scaling application performance across it. Paired with NVIDIA Quantum InfiniBand, HGX delivers world-class performance and efficiency, which ensures the full utilization of computing resources.

For AI cloud data centers that deploy Ethernet, HGX is best used with the NVIDIA Spectrum-X networking platform, which powers the highest AI performance over 400Gb/s Ethernet. Featuring NVIDIA Spectrum™-4 switches and BlueField-3 DPUs, Spectrum-X delivers consistent, predictable outcomes for thousands of simultaneous AI jobs at every scale through optimal resource utilization and performance isolation. Spectrum-X enables advanced cloud multi-tenancy and zero-trust security.  As a reference design for NVIDIA Spectrum-X, NVIDIA has designed Israel-1, a hyperscale generative AI supercomputer built with Dell PowerEdge XE9680 servers based on the NVIDIA HGX™ H100 eight-GPU platform, BlueField-3 DPUs, and Spectrum-4 switches.

Connecting HGX H200 or HGX H100 with NVIDIA Networking

  NVIDIA Quantum-2 InfiniBand Platform:

Quantum-2 Switch, ConnectX-7 Adapter, BlueField-3 DPU

 NVIDIA Spectrum-X Platform:

Spectrum-4 Switch, BlueField-3 DPU, Spectrum-X License

 NVIDIA Spectrum Ethernet Platform:

Spectrum Switch, ConnectX Adapter, BlueField DPU
DL Training BEST BETTER GOOD
Scientific Sim BEST BETTER GOOD
Data Analytics BEST BETTER GOOD
DL Inference BEST BETTER GOOD

NVIDIA HGX Specifications

NVIDIA HGX is available in single baseboards with four or eight H200 or H100 GPUs, or four or eight A100 GPUs. These powerful combinations of hardware and software lay the foundation for unprecedented AI supercomputing performance.

  HGX H200
  4-GPU 8-GPU
GPUs HGX H200 4-GPU HGX H200 8-GPU
Form factor 4x NVIDIA H200 SXM 8x NVIDIA H200 SXM
HPC and AI compute (FP64/TF32/FP16/FP8/INT8) 268TF/4PF/8PF/16PF/16 POPS 535TF/8PF/16PF/32PF/32 POPS
Memory Up to 564GB Up to 1.1TB
NVLink Fourth generation Fourth generation
NVSwitch N/A Third generation
NVSwitch GPU-to-GPU bandwidth N/A 900GB/s
Total aggregate bandwidth 3.6TB/s 7.2TB/s
  HGX H100
  4-GPU 8-GPU
GPUs HGX H100 4-GPU HGX H100 8-GPU
Form factor 4x NVIDIA H100 SXM 8x NVIDIA H100 SXM
HPC and AI compute (FP64/TF32/FP16/FP8/INT8) 268TF/4PF/8PF/16PF/16 POPS 535TF/8PF/16PF/32PF/32 POPS
Memory Up to 320GB Up to 640GB
NVLink Fourth generation Fourth generation
NVSwitch N/A Third generation
NVLink Switch N/A N/A
NVSwitch GPU-to-GPU bandwidth N/A 900GB/s
Total aggregate bandwidth 3.6TB/s 7.2TB/s
  HGX A100
  4-GPU 8-GPU
GPUs HGX A100 4-GPU HGX A100 8-GPU
Form factor 4x NVIDIA A100 SXM 8x NVIDIA A100 SXM
HPC and AI compute (FP64/TF32/FP16/INT8) 78TF/1.25PF/2.5PF/5 POPS 156TF/2.5PF/5PF/10 POPS
Memory Up to 320GB Up to 640GB
NVLink Third generation Third generation
NVSwitch N/A Second generation
NVSwitch GPU-to-GPU bandwidth N/A 600GB/s
Total aggregate bandwidth 2.4TB/s 4.8TB/s

* With sparsity

Find out more about the NVIDIA H200 Tensor Core GPU.

Learn More
Products
Technologies
Software
Resources