Generative AI Framwork

NVIDIA NeMo
Framework

Build, customize, and deploy generative AI models.

Download Now

    What is the NVIDIA NeMo Framework?

    NVIDIA NeMo™ framework, part of the NVIDIA AI platform, is an end-to-end, cloud-native enterprise framework to build, customize, and deploy generative AI models with billions of parameters.

    The NeMo framework provides an accelerated workflow for training with 3D parallelism techniques. It offers a choice of several customization techniques and is optimized for at-scale inference of large-scale models for language and image applications, with multi-GPU and multi-node configurations.NeMo makes generative AI model development easy, cost-effective, and fast for enterprises.

    How the NeMo Framework builds, trains, and deploys large language models.

    Build Foundation Models for Different Modalities

    The NeMo framework supports the development of text-to-text, text-to-image, and image-to-image foundation models.

    Language

    • BERT
    • GPT-3
    • T5
    • T5-MoE
    • Inform

    Multimodal

    •  Stable Diffusion v1.5
    •  VisionTransformers (ViT) 
    •  CLIP
    •  Instruct-Pix2Pix
    •  Imagen

    Build Trustworthy, Safe, and Secure LLM Applications

    Programmable Guardrails for LLM-Based Applications

    NeMo Guardrails is a toolkit for easily developing trustworthy, safe and secure LLM conversational systems. It natively supports LangChain, adding a layer of safety, security, and topical guardrails to LLM-based conversational applications.

    Read Technical Blog
    Try NeMo Guardrails Now

    Explore the Benefits

    Fastest Training on GPUs

    Use state-of-the-art training techniques to maximize throughput and minimize training time for foundation models with billions or trillions of parameters.

    Easy to Use

    A cloud-native framework with all dependencies pre-packaged and installed with validated receipts for training language and image generative AI models to convergence and deploy for inference.

    Fully Flexible

    An open-source approach offering full flexibility across the pipeline—from data processing, to training, to inference of generative AI models.

    Run in the Cloud and On Premises

    Train and deploy foundation models of any size on any GPU infrastructure. Supported on all NVIDIA DGX™ systems, NVIDIA DGX Cloud™, Microsoft Azure, Oracle Cloud Infrastructure, and Amazon Web Services.

    Customize

    Offers tools to customize foundation models for enterprise hyper-personalization.

    Enterprise Support

    Battle hardened, tested, and verified containers built for enterprises.

    Try the NeMo Framework Through a Free, Hands-On Labs on NVIDIA LaunchPad

    Try Now

    Key Features to Develop Large Language Models

    State-of-the-Art Training Techniques

    The NeMo framework delivers high levels of training efficiency, making training of large-scale foundation models possible, using 3D parallelism techniques such as:

    • Tensor parallelism to scale models within nodes
    • Data and pipeline parallelism to scale data and models across thousands of GPUs 
    • Sequence parallelism to distribute activation memory across tensor parallel devices

    In addition, selective activation recomputing optimizes recomputation and memory usage across tensor parallel devices during backpropagation.

    Read Blog

    Customization Tools

    The NeMo framework makes enterprise AI practical by offering tools to:

    • Define focus and guardrails: Define guardrails and the operating domain for hyper-personalized enterprise models to prevent LLMs from veering off into unwanted domains or saying inappropriate things, through fine-tuning, prompt learning, and adapter techniques. 
    • Include domain-specific knowledge: Encode and embed your AI with your enterprise’s real-time information to provide the latest responses, using NVIDIA Inform.
    • Include functional skills: Add specialized skills to solve customer and business problems. Get better responses by providing context for hyper-personalized use cases using prompt learning techniques. 
    • Continuously improve the model: Reinforcement learning with human feedback (RLHF) techniques allow your enterprise model to get smarter over time, aligned with human intentions.
    Read Blog

    Optimized AI Inference With NVIDIA Triton

    Deploy generative AI models for inference using NVIDIA Triton Inference Server™. With powerful optimizations from FasterTransformer, you can achieve state-of-the-art accuracy, latency, and throughput inference performance on single-GPU, multi-GPU, and multi-node configurations.

    Learn More About Triton

    Data Processing at Scale

    Bring your own dataset and tokenize data to a digestible format. NeMo includes comprehensive preprocessing capabilities for data filtration, deduplication, blending, and formatting on language datasets, on Piles and multilingual C4 (mC4). These help researchers and engineers save months of development and compute time, letting them focus on building applications.

    Easy-to-Use Recipes and Tools for Generative AI

    The NeMo framework makes generative AI possible from day one with prepackaged scripts, reference examples, and documentation across the entire pipeline.

    Building foundation models is also made easy through an auto-configurator tool, which automatically searches for the best hyperparameter configurations to optimize training and inference for any given multi-GPU configuration, training, or deployment constraints.

    Easily Customize and Use Generative AI models using NVIDIA NeMo Language and Image Services

    NeMo Language Service

    Cloud service for enterprise hyper-personalization and at-scale deployment of intelligent large language models.

    Learn more about NeMo LLM service Sign up for Early Access Now

    Picasso Service

    An accelerated cloud service for enterprises that uses custom generative AI models for creating high-res, photorealistic images, videos, and 3D content.

    Learn About NVIDIA Picasso Get Notified When Available

    Customers Accelerating Generative AI and LLM Applications with NVIDIA NeMo Framework

    Accelerate industry applications With LLMs.

    AI Sweden accelerated LLM industry applications by making the power of a 100 billion parameter model for regional languages easily accessible to the Nordic ecosystem. AI Sweden is digitizing Sweden’s historical records and building language models from this unstructured data that can be commercialized in enterprise applications.

    Image Courtesy of Korea Telecom

    Creating New Customer Experiences with LLMs

    South Korea’s leading mobile operator builds billion-parameter large language models trained with the NVIDIA DGX SuperPOD platform and NeMo framework to power smart speakers, and customer call centers

    Discover more resources

    Deploying a 1.3B GPT-3 Model With NVIDIA NeMo Framework

    Learn how to download, optimize, and deploy a 1.3 billion parameter GPT-3 model with the NeMo framework and NVIDIA’s generative AI framework.

    Read the Blog

    Efficient At-Scale Training and Deployment of LLMs With NeMo Framework

    Learn how to preprocess data in a multi-node environment, automatically select the best hyperparameters to minimize training time for multiple GPT-3 and T5 configurations, train the model at scale, and deploy the model in a multi-node production setting with an easy-to-use set of scripts.

    Watch Video

    Free Hands-On Lab on NVIDIA LaunchPad

    Bootstrap your enterprise's LLM journey using pretuned hyperparameter configurations for GPT-3 models. Learn how to train a large-scale NLP model with NeMo framework.

    Register for the Trial

    Get Started Now with NVIDIA NeMo Framework

    Get Started
    Products
    Technologies
    Software
    Resources