Setting a New Bar in MLPerf

NVIDIA training and inference solutions deliver record-setting performance in MLPerf, the leading industry benchmark for AI performance.

See Our Results

What is MLPerf?

MLPerf is a consortium of AI leaders from academia, research labs, and industry whose mission is to “build fair and useful benchmarks” that provide unbiased evaluations of training and inference performance for hardware, software, and services—all conducted under prescribed conditions. To stay on the cutting edge of industry trends, MLPerf continues to evolve, holding new tests at regular intervals and adding new workloads that represent the state of the art in AI.

MLPerf Submission Categories

MLPerf Training v0.7 is the third instantiation for training and consisted of eight different workloads covering a broad diversity of use cases, including vision, language, recommenders, and reinforcement learning.

MLPerf Inference v0.7 tested five different use-cases across seven different kinds of neural networks. Three of these use cases were for computer vision, one for recommender systems, two for language processing and one for medical imaging.

Image Classification

Image Classification

Assigns a label from a fixed set of categories to an input image, i.e., applies to computer vision problems such as autonomous vehicles. details.

Object Detection (Lightweight)

Object Detection (Lightweight)

Finds instances of real-world objects such as faces, bicycles, and buildings in images or videos and specifies a bounding box around each. details.

Object Detection (Heavyweight)

Object Detection (Heavyweight)

Detects distinct objects of interest appearing in an image and identifies a pixel mask for each. details.

Translation (Recurrent)

Translation (Recurrent)

Translates text from one language to another using a recurrent neural network (RNN). details.

Translation (Non-recurrent)

Translation (Non-recurrent)

Translates text from one language to another using a feed-forward neural network. details.

Natural Language Processing (NLP)

Natural Language Processing (NLP)

Understands text by using the relationship between different words in a block of text. Allows for question answering, sentence paraphrasing, and many other language-related use cases. details.

Recommendation

Recommendation

Delivers personalized results in user-facing services such as social media or e-commerce websites by understanding interactions between users and service items, like products or ads. details.

Reinforcement Learning

Reinforcement Learning

Evaluates different possible actions to maximize reward using the strategy game Go played on a 19x19 grid. details.

NVIDIA’s MLPerf Benchmark Results

  • Training

    Training

  • Inference

    Inference

The NVIDIA A100 Tensor Core GPU and the NVIDIA DGX SuperPOD™ set all 16 training performance records, both in per-chip and at-scale workloads for commercially available systems. This breakthrough performance came from the tight integration of hardware, software, and system-level technologies. NVIDIA’s continuous investment in full-stack performance has led to an improvement in throughput over the three MLPerf submissions.

UP TO 4X THE PERFORMANCE IN 1.5 YEAR​S OF MLPERF

NVIDIA's Full Stack Innovation Delivers Continuous Improvements

UP TO 4X THE PERFORMANCE IN 1.5 YEAR​S OF MLPERF

Comparing the throughput of a single DGX-1V server on a single epoch and DGX A100 (single pass of the dataset through the neural network) | MLPerf ID 0.5/0.6/0.7 comparison: ResNet-50 v1.5: 0.5-12, /0.6-8 /0.7-39, 0.7-18 | SSD: 0.5-13/0.6-9 /0.7-40, 0.719 | GNMT: 0.5-13/0.6-9 /0.7-40 0.7-19 | Mask R-CNN: 0.5-13/0.6-9 /0.7-40 0.7-19 | Transformer: 0.5-13/0.6-9 0.7-40, 0.7-19. MLPerf name and logo are trademarks. See www.mlperf.org for more information.

NVIDIA sets all 16 records

For Commercially Available Solutions

  Max Scale Records Per-Accelerator Records
Recommendation (DLRM) 3.33 min 0.44 hrs
NLP (BERT) 0.81 min 6.53 hrs
Reinforcement Learning (MiniGo) 17.07 min 39.96 hrs
Translation (Non-recurrent) (Transformer) 0.62 min 1.05 hrs
Translation (Recurrent) (GNMT) 0.71 min 1.04 hrs
Object Detection (Heavyweight) (Mask R-CNN) 10.46 min 10.95 hrs
Object Detection (Lightweight) (SSD) 0.82 min 1.36 hrs
Image Classification (ResNet-50 v1.5) 0.76 min 5.30 hrs

Per-chip performance arrived at by comparing performance at the same scale when possible. Per-accelerator comparison using reported performance for MLPerf 0.7 NVIDIA A100 (8x A100s). MLPerf ID DLRM: 0.7-17, ResNet50 v1.5: 0.7-18, 0.7-15 BERT, GNMT, Mask R-CNN, SSD, Transformer: 07-19, MiniGo: 0.7-20. Max scale: All results from MLPerf v0.7 using NVIDIA DGX A100 (8x A100s). MLPerf ID Max Scale: ResNet-50 v1.5: 0.7-37, Mask R-CNN: 0.7-28, SSD: 0.7-33, GNMT: 0.7-34, Transformer: 0.7-30, MiniGo: 0.7-36, BERT: 0.7-38, DLRM: 0.7-17. | MLPerf name and logo are trademarks. See www.mlperf.org for more information.

NVIDIA achieved top results in all five scenarios (data center server and offline, as well as edge single-stream, multi-stream, and offline). In addition, we delivered the best per-accelerator performance among all products tested across all five benchmark tests. These results are a testament, not only to NVIDIA's inference performance leadership, but to the versatility of our inference platform.

Server scenario for data center and edge

  NVIDIA T4
(Inferences/Second)		 NVIDIA A100
(Inferences/Second)		 NVIDIA® Jetson Xavier™
(Max Inferences/Query)
DLRM
(Recommender)		 34,052 264,189 N/A
BERT
(Natural Language Processing)		 437 3,328 91
ResNet-50 v1.5
(Image Classification)		 6,112 37,331 2,075
MobileNet-v1
(Small Single Shot Detector)		 995 6,401 2,533
ResNet-34
(Large Single Shot Detector)		 139 974 51
RNN-T
(Speech Recognition)		 1,495 10,300 419
3D U-Net
(Medical Imaging)		 7 42 2.3

1) MLPerf v0.7 Inference results for data center server form factors and offline and server scenarios retrieved from www.mlperf.org on Oct. 21, 2020. Per-processor performance is calculated by dividing the primary metric of total performance by the number of accelerators reported.
2) MLPerf v0.7 Inference results for edge form factors and single-stream and multi-stream scenarios retrieved from www.mlperf.org on Oct. 21, 2020.

The Technology Behind the Results

The complexity of AI demands a tight integration between all aspects of the platform. As demonstrated in MLPerf's benchmarks, the NVIDIA AI platform delivers leadership performance with the world's most advanced GPU, powerful and scalable interconnect technologies, and cutting-edge software—an end-to-end solution that can be deployed in the data center, in the cloud, or at the edge with amazing results.

Optimized Software that Accelerates AI Workflows

Optimized Software that Accelerates AI Workflows

An essential component of NVIDIA’s platform and MLPerf training and inference results, NGC™ is a hub for GPU-optimized AI, high-performance computing (HPC), and data analytics software that simplifies and accelerates end-to-end workflows. With over 150 enterprise-grade containers, including workloads for Conversational AI and recommender systems, over 100 models, and industry-specific SDKs that can be deployed on-premises, in the cloud, or at the edge, NGC enables data scientists, researchers, and developers to build best-in-class solutions, gather insights, and deliver business value faster than ever before.

VISIT NGC

Leadership-Class AI Infrastructure

Achieving world-leading results across training and inferences requires infrastructure that’s purpose-built for the world’s most complex AI challenges. The NVIDIA AI platform delivered using the power of the NVIDIA A100 Tensor Core GPU, the NVIDIA T4 Tensor Core GPU, and the scalability and flexibility of NVIDIA interconnect technologies—NVLink®, NVSwitch™, and the Mellanox ConnectX-6 VPI. These are at the heart of the NVIDIA DGX™ A100, the engine behind our benchmark performance.

NVIDIA DGX™ systems offer the scalability, rapid deployment, and incredible compute power that can enable every enterprise to build leadership-class AI infrastructure.

LEARN MORE
Leadership-Class AI Infrastructure

Learn more about our data center training and inference product performance.

View Performance
Image Classification

Image Classification

MLPerf Training and Inference used ResNet v1.5 with the ImageNet dataset.
MLPerf Inference used ResNet-50 v. 1.5 and MobileNet-v1 with the ImageNet dataset.
Object Detection (Lightweight)

Object Detection (Lightweight)

MLPerf Training used Single-Shot Detector (SSD) on the COCO dataset.
MLPerf Inference used SSD-MobileNet-v1 (low-res, 0.09 MP), with the COCO dataset.
Object Detection (Heavyweight)

Object Detection (Heavyweight)

MLPerf Training used Mask R-CNN on the COCO dataset.
MLPerf Inference used SSD-ResNet-34 (high-res, 1.44 MP) with the COCO dataset.
Translation (Recurrent)

Translation (Recurrent)

MLPerf used Google’s Machine Translation System (GNMT) on the WMT English-German dataset.
MLPerf Inference used the Google Neural Machine Translation (GNMT) RNN with the WMT16 dataset.
Translation (Non-recurrent)

Translation (Non-recurrent)

MLPerf used Transformer on the WMT English-German dataset.
MLPerf Inference measured speech recognition using the RNN-T network with the LibriSpeech dataset.
Natural Language Processing (NLP)

Natural Language Processing (NLP)

MLPerf used Bidirectional Encoder Representations from Transformers (BERT) on the Wikipedia 2020/01/01 dataset.
MLPerf Inference used BERT with the SQuAD v.1.1 dataset.
Recommendation

Recommendation

MLPerf used the Deep Learning Recommendation Model (DLRM) on 1 TB Click Logs dataset.
MLPerf Inference used the DLRM network with the Criteo Terabyte dataset.
Reinforcement Learning

Reinforcement Learning

MLPerf used Mini Go based on the Go dataset.
Products
Technologies
Software
Resources