This site requires Javascript in order to view all its content. Please enable Javascript in order to access all the functionality of this web site. Here are the instructions how to enable JavaScript in your web browser.

Instructor-Led Workshop
Accelerating CUDA C++ Applications with Multiple GPUs

Request a workshop for your organization

Notify me when public workshops are available

View public workshops

Computationally intensive CUDA^® C++ applications in high-performance computing, data science, bioinformatics, and deep learning can be accelerated by using multiple GPUs, which can increase throughput and/or decrease your total runtime. When combined with the concurrent overlap of computation and memory transfers, computation can be scaled across multiple GPUs without increasing the cost of memory transfers. For organizations with multi-GPU servers, whether in the cloud or on NVIDIA DGX^™ systems, these techniques enable you to achieve peak performance from GPU-accelerated applications. And it’s important to implement these single-node, multi-GPU techniques before scaling your applications across multiple nodes.

This workshop covers how to write CUDA C++ applications that efficiently and correctly utilize all available GPUs in a single node, dramatically improving the performance of your applications and making the most cost-effective use of systems with multiple GPUs.

Learning Objectives

By participating in this workshop, you’ll:

Use concurrent CUDA streams to overlap memory transfers with GPU computation
Utilize all available GPUs on a single node to scale workloads across all available GPUs
Combine the use of copy/compute overlap with multiple GPUs
Rely on the NVIDIA Nsight^™ Systems Visual Profiler timeline to observe improvement opportunities and the impact of the techniques covered in the workshop

Download workshop datasheet (PDF 243 KB)

Workshop Outline

Introduction (15 mins)	Meet the instructor. Create an account at courses.nvidia.com/join
Using JupyterLab (15 mins)	Get familiar with your GPU-accelerated interactive JupyterLab environment.
Application Overview (15 mins)	Orient yourself with a single GPU CUDA C++ application that will be the starting point for the course. Observe the current performance of the single GPU CUDA C++ application using Nsight Systems.
Introduction to CUDA Streams (90 mins)	Learn the rules that govern concurrent CUDA stream behavior. Use multiple CUDA streams to perform concurrent host-to-device and device-to-host memory transfers. Utilize multiple CUDA streams for launching GPU kernels. Observe multiple streams in the Nsight Systems Visual Profiler timeline view.
Break (60 mins)
Copy/Compute Overlap with CUDA Streams (90 mins)	Learn the key concepts for effectively performing copy/compute overlap. Explore robust indexing strategies for the flexible use of copy/compute overlap in applications. Refactor the single-GPU CUDA C++ application to perform copy/compute overlap. See copy/compute overlap in the Nsight Systems visual profiler timeline.
Multiple GPUs with CUDA C++ (60 mins)	Learn the key concepts for effectively using multiple GPUs on a single node with CUDA C++. Explore robust indexing strategies for the flexible use of multiple GPUs in applications. Refactor the single-GPU CUDA C++ application to utilize multiple GPUs. See multiple-GPU utilization in the Nsight Systems Visual Profiler timeline.
Break (15 mins)
Copy/Compute Overlap with Multiple GPUs (60 mins)	Learn the key concepts for effectively performing copy/compute overlap on multiple GPUs. Explore robust indexing strategies for the flexible use of copy/compute overlap on multiple GPUs. Refactor the single-GPU CUDA C++ application to perform copy/compute overlap on multiple GPUs. Observe performance benefits for copy/compute overlap on multiple GPUs. See copy/compute overlap on multiple GPUs in the Nsight Systems visual profiler timeline.
Course Assessment (30 mins)
Final Review (30 mins)	Review key learnings. Learn to build your own training environment from the DLI base environment container. Complete the workshop survey.

Workshop Details

Duration: 8 hours

Price: Contact us for pricing.

Prerequisites:

Professional experience programming CUDA C/C++ applications, including the use of the nvcc compiler, kernel launches, grid-stride loops, host-to-device and device-to-host memory transfers, and CUDA error handling
Familiarity with the Linux command line
Experience using makefiles to compile C/C++ code

Suggested resources to satisfy prerequisites: Fundamentals of Accelerated Computing with CUDA C/C++, Ubuntu Command Line for Beginners (sections 1 through 5), Makefile Tutorial (through the Simple Examples section)

Technologies: CUDA C++, Nsight Systems

Certificate: Upon successful completion of the assessment, participants will receive an NVIDIA DLI certificate to recognize their subject matter competency and support professional career growth.

Hardware Requirements: Desktop or laptop computer capable of running the latest version of Chrome or Firefox. Each participant will be provided with dedicated access to a fully configured, GPU-accelerated server in the cloud.

Languages: English, Simplified Chinese

Upcoming Workshops

Central European Time

GTC

Thu, Apr 15, 2021

9:00 a.m.–5:00 p.m.

Register now

Pacific Time

GTC

Fri, Apr 16, 2021

9:00 a.m.–5:00 p.m.

Register now

Pacific Time

Wed, Jun 23, 2021

9:00 a.m.–5:00 p.m.

Register now

If your organization is interested in boosting and developing key skills in AI, accelerated data science, or accelerated computing, you can request instructor-led training from the NVIDIA DLI.

Request a Workshop

Questions?

Contact Us for Questions on
Deep Learning Training

Section

Section

First Name

Last Name

Business Email Address

Organization / University Name

Industry

Job Title

Location

City

State

Reason for requesting contact

NVIDIA Privacy Policy

I agree to the collection and processing of the above information by NVIDIA Corporation for the purposes of research and event organization, and I have read and agree to <a href="https://www.nvidia.com/en-gb/about-nvidia/privacy-policy/?deeplink=visiting-our-website" target="_blank">NVIDIA Privacy Policy</a>.

I agree that the above information will be transferred to NVIDIA Corporation in the United States and stored in a manner consistent with <a href="https://www.nvidia.com/en-gb/about-nvidia/privacy-policy/?deeplink=visiting-our-website" target="_blank">NVIDIA Privacy Policy</a> due to necessities for research, event organization and corresponding NVIDIA internal management and system operation need. You may contact us by sending an email to <a href="mailto:privacy@nvidia.com">privacy@nvidia.com</a> to resolve related problems.

Instructor-Led Workshop
Accelerating CUDA C++ Applications with Multiple GPUs

Learning Objectives

Workshop Outline

Workshop Details

Upcoming Workshops

Thu, Apr 15, 2021

Fri, Apr 16, 2021

Wed, Jun 23, 2021

Questions?

Contact Us for Questions on
Deep Learning Training

Read our FAQs.

Inquire about NVIDIA Deep Learning Institute services.

For technical questions, check out the NVIDIA Developer Forums.

Instructor-Led Workshop Accelerating CUDA C++ Applications with Multiple GPUs

Learning Objectives

Workshop Outline

Workshop Details

Upcoming Workshops

Thu, Apr 15, 2021

Fri, Apr 16, 2021

Wed, Jun 23, 2021

Questions?

Contact Us for Questions on Deep Learning Training

Read our FAQs.

Inquire about NVIDIA Deep Learning Institute services.

For technical questions, check out the NVIDIA Developer Forums.

Instructor-Led Workshop
Accelerating CUDA C++ Applications with Multiple GPUs

Contact Us for Questions on
Deep Learning Training