NVIDIA GH200 Grace Hopper Superchip

Bringing together the Arm®-based Grace CPU and Hopper GPU architectures using NVLink-C2C, NVIDIA GH200 Superchips are speeding up scientific research and discovery at supercomputers worldwide. Combined, supercomputers leveraging NVIDIA Grace Hopper Superchips deliver 200 exaflops, or 200 quintillion calculations per second, of energy-efficient AI processing power.

Recent advancements in storage and networking bandwidth, along with the end of Moore’s law, have shifted analytics and query bottlenecks to CPUs. With GH200, the CPU and GPU share a single per-process page table, enabling all CPU and GPU threads to access all system-allocated memory that can reside on physical CPU or GPU memory. GH200 removes the need to copy memory back and forth between the CPU and GPU, accelerating data processing by up to 36X.

Retrieval-augmented generation (RAG) connects large language models (LLMs) with knowledge bases to improve model accuracy. RAG requires embeddings to be generated and vector searches to be executed at scale. The 72 power-efficient Arm-based Grace CPU cores accelerate preprocessing of the knowledge base data, and NVLink-C2C speeds up transfer of the preprocessed data to the Hopper GPU by 7X compared to PCIe—resulting in a 30X speedup of the embedding generation process.

Graph neural networks (GNNs) are used in a wide range of applications, such as social network analysis, drug discovery, fraud detection, and molecular chemistry. GH200 takes advantage of the up to 624GB of combined CPU and GPU fast memory, 4 petaFLOPS of H200 GPU, and the 900 GB/s NVLink-C2C to speed up GNN training by up to 8X compared to the H100 PCIe GPU.