This site requires Javascript in order to view all its content. Please enable Javascript in order to access all the functionality of this web site. Here are the instructions how to enable JavaScript in your web browser.

Instructor-Led Workshop
Fundamentals of Deep Learning for Multi-GPUs

Request a workshop for your organization

Notify me when public workshops are available

View public workshops

This workshop teaches you techniques for training deep neural networks on multi-GPU technology to shorten the training time required for data-intensive applications. Working with deep learning tools, frameworks, and workflows to perform neural network training, you’ll learn concepts for implementing Horovod multi-GPUs to reduce the complexity of writing efficient distributed software and to maintain accuracy when training a model across many GPUs.

Learning Objectives

At the conclusion of the workshop, you’ll have an understanding of:

Stochastic gradient descent (SGD), a crucial tool in parallelized training
Batch size and its effect on training time and accuracy
Transforming a single-GPU implementation to a Horovod multi-GPU implementation
Techniques for maintaining high accuracy when training across multiple GPUs

Download workshop datasheet (PDF 73.3 KB)

Workshop Outline

Introduction (15 mins)	Meet the instructor. Create an account at courses.nvidia.com/join
Stochastic Gradient Descent and the Effects of Batch Size (120 mins)	Understand the issues with sequential single-thread data processing and the theory behind speeding up applications with parallel processing. Explore loss function, gradient descent, and SGD. Learn the effect of batch size on accuracy and training time.
Break (60 mins)
Training on Multiple GPUs with Horovod (120 mins)	Discover the benefits of training on multiple GPUs with Horovod. Learn to transform single-GPU training on the Fashion-MNIST dataset to Horovod multi-GPU implementation.
Break (15 mins)
Maintaining Model Accuracy when Scaling to Multiple GPUs (120 mins)	Understand why accuracy can decrease when parallelizing training on multiple GPUs. Explore tools for maintaining accuracy when scaling training to multiple GPUs.
Final Review (15 mins)	Review key learnings and answer questions. Complete the assessment and earn a certificate. Complete the workshop survey. Learn how to set up your own AI application development environment.

Workshop Details

Duration: 8 hours

Price: Contact us for pricing.

Prerequisites: Experience with gradient descent model training

Technologies: TensorFlow, Keras, Horovod

Assessment Type: Code-based

Certificate: Upon successful completion of the assessment, participants will receive an NVIDIA DLI certificate to recognize their subject matter competency and support professional career growth.

Hardware Requirements: Desktop or laptop computer capable of running the latest version of Chrome or Firefox. Each participant will be provided with dedicated access to a fully configured, GPU-accelerated server in the cloud.

Languages: English, Korean, Simplified Chinese, Traditional Chinese

Upcoming Workshops

Pacific Time

GTC

Thu, Apr 15, 2021

9:00 a.m.–5:00 p.m.

Register now

If your organization is interested in boosting and developing key skills in AI, accelerated data science, or accelerated computing, you can request instructor-led training from the NVIDIA DLI.

Request a Workshop

Questions?

Contact Us for Questions on
Deep Learning Training

Section

Section

First Name

Last Name

Business Email Address

Organization / University Name

Industry

Job Role

Job Title

Location

City

State

Reason for requesting contact

本人知悉且同意 NVIDIA 基于调研、活动组织的目的对本人的上述信息的收集和处理，并已经阅读并同意 <a href="https://www.nvidia.cn/about-nvidia/privacy-policy/" target="_blank">NVIDIA 隐私政策</a>。

本人知悉且同意，因调研、活动组织的必须，以及相对应的 NVIDIA 内部管理和系统操作的需要，上述信息会被传输到位于美国的 NVIDIA Corporation 按照符合 <a href="https://www.nvidia.cn/about-nvidia/privacy-policy/" target="_blank">NVIDIA 隐私政策</a>的方式进行存储，您可以通过发送邮件至 <a href="mailto:privacy@nvidia.com">privacy@nvidia.com</a> 进行联系以解决相关问题，实现可适用的数据保护法所规定的权利。

Instructor-Led Workshop
Fundamentals of Deep Learning for Multi-GPUs

Learning Objectives

Workshop Outline

Workshop Details

Upcoming Workshops

Thu, Apr 15, 2021

Questions?

Contact Us for Questions on
Deep Learning Training

Read our FAQs.

Inquire about NVIDIA Deep Learning Institute services.

For technical questions, check out the NVIDIA Developer Forums.

Instructor-Led Workshop Fundamentals of Deep Learning for Multi-GPUs

Learning Objectives

Workshop Outline

Workshop Details

Upcoming Workshops

Thu, Apr 15, 2021

Questions?

Contact Us for Questions on Deep Learning Training

Read our FAQs.

Inquire about NVIDIA Deep Learning Institute services.

For technical questions, check out the NVIDIA Developer Forums.

Instructor-Led Workshop
Fundamentals of Deep Learning for Multi-GPUs

Contact Us for Questions on
Deep Learning Training