This site requires Javascript in order to view all its content. Please enable Javascript in order to access all the functionality of this web site. Here are the instructions how to enable JavaScript in your web browser.

Instructor-Led Workshop
Fundamentals of Deep Learning for Multi-GPUs

Request a workshop for your organization

Notify me when public workshops are available

View public workshops

This workshop teaches you techniques for training deep neural networks on multi-GPU technology to shorten the training time required for data-intensive applications. Working with deep learning tools, frameworks, and workflows to perform neural network training, you’ll learn concepts for implementing Horovod multi-GPUs to reduce the complexity of writing efficient distributed software and to maintain accuracy when training a model across many GPUs.

Learning Objectives

At the conclusion of the workshop, you’ll have an understanding of:

Stochastic gradient descent (SGD), a crucial tool in parallelized training
Batch size and its effect on training time and accuracy
Transforming a single-GPU implementation to a Horovod multi-GPU implementation
Techniques for maintaining high accuracy when training across multiple GPUs

Download workshop datasheet (PDF 73.3 KB)

Workshop Outline

Introduction (15 mins)	Meet the instructor. Create an account at courses.nvidia.com/join
Stochastic Gradient Descent and the Effects of Batch Size (120 mins)	Understand the issues with sequential single-thread data processing and the theory behind speeding up applications with parallel processing. Explore loss function, gradient descent, and SGD. Learn the effect of batch size on accuracy and training time.
Break (60 mins)
Training on Multiple GPUs with Horovod (120 mins)	Discover the benefits of training on multiple GPUs with Horovod. Learn to transform single-GPU training on the Fashion-MNIST dataset to Horovod multi-GPU implementation.
Break (15 mins)
Maintaining Model Accuracy when Scaling to Multiple GPUs (120 mins)	Understand why accuracy can decrease when parallelizing training on multiple GPUs. Explore tools for maintaining accuracy when scaling training to multiple GPUs.
Final Review (15 mins)	Review key learnings and answer questions. Complete the assessment and earn a certificate. Complete the workshop survey. Learn how to set up your own AI application development environment.

Workshop Details

Duration: 8 hours

Price: Contact us for pricing.

Prerequisites: Experience with gradient descent model training

Technologies: TensorFlow, Keras, Horovod

Assessment Type: Code-based

Certificate: Upon successful completion of the assessment, participants will receive an NVIDIA DLI certificate to recognize their subject matter competency and support professional career growth.

Hardware Requirements: Desktop or laptop computer capable of running the latest version of Chrome or Firefox. Each participant will be provided with dedicated access to a fully configured, GPU-accelerated server in the cloud.

Languages: English, Korean, Simplified Chinese, Traditional Chinese

Upcoming Workshops

Pacific Time

GTC

Thu, Apr 15, 2021

9:00 a.m.–5:00 p.m.

Register now

If your organization is interested in boosting and developing key skills in AI, accelerated data science, or accelerated computing, you can request instructor-led training from the NVIDIA DLI.

Request a Workshop

Questions?

Contact Us for Questions on
Deep Learning Training

Section

Section

First Name

Last Name

Business Email Address

Organization/University Name

Industry

Job Title

Location

City

State

Reason for requesting contact

nvid hidden field

ncid hidden field

NVIDIA Privacy Policy

I agree to the collection and processing of the above information by NVIDIA Corporation for the purposes of research and event organization, and I have read and agree to <a href="https://www.nvidia.com/en-us/about-nvidia/privacy-policy/?deeplink=visiting-our-website" target="_blank">NVIDIA Privacy Policy</a>.

I agree that the above information will be transferred to NVIDIA Corporation in the United States and stored in a manner consistent with <a href="https://www.nvidia.com/en-us/about-nvidia/privacy-policy/?deeplink=visiting-our-website" target="_blank">NVIDIA Privacy Policy</a> due to necessities for research, event organization and corresponding NVIDIA internal management and system operation need. You may contact us by sending an email to <a href="mailto:privacy@nvidia.com">privacy@nvidia.com</a> to resolve related problems.

Instructor-Led Workshop
Fundamentals of Deep Learning for Multi-GPUs

Learning Objectives

Workshop Outline

Workshop Details

Upcoming Workshops

Thu, Apr 15, 2021

Questions?

Contact Us for Questions on
Deep Learning Training

Read our FAQs.

Inquire about NVIDIA Deep Learning Institute services.

For technical questions, check out the NVIDIA Developer Forums.

Instructor-Led Workshop Fundamentals of Deep Learning for Multi-GPUs

Learning Objectives

Workshop Outline

Workshop Details

Upcoming Workshops

Thu, Apr 15, 2021

Questions?

Contact Us for Questions on Deep Learning Training

Read our FAQs.

Inquire about NVIDIA Deep Learning Institute services.

For technical questions, check out the NVIDIA Developer Forums.

Instructor-Led Workshop
Fundamentals of Deep Learning for Multi-GPUs

Contact Us for Questions on
Deep Learning Training