NVIDIA Run:ai

The enterprise platform for AI workloads and GPU orchestration.

Get Started

Documentation | Solution Overview | GitHub

Overview
Platform
Performance
Open Source
Benefits
Adoption Options
Ecosystem
Next Steps

Overview
Platform
Performance
Open Source
Benefits
Adoption Options
Ecosystem
Next Steps

Get Started

Overview

Accelerate AI Workflows With Dynamic Orchestration

NVIDIA Run:ai accelerates AI and machine learning operations by addressing key infrastructure challenges through dynamic resource allocation, comprehensive AI life cycle support, and strategic resource management. By pooling resources across environments and utilizing advanced orchestration, NVIDIA Run:ai significantly enhances GPU efficiency and workload capacity. With support for public clouds, private clouds, hybrid environments, or on-premises data centers, NVIDIA Run:ai provides unparalleled flexibility and adaptability.

Build With NVIDIA Run:ai

Adopt the complete platform or build with one or more open source NVIDIA Run:ai projects.

Explore NVIDIA Run:ai on GitHub

What Is NVIDIA Run:ai?

NVIDIA Run:ai is the open source AI workload and GPU orchestration platform for Kubernetes. It enables organizations to run training, fine-tuning, and inference at scale while maximizing GPU efficiency and governing shared infrastructure.

NVIDIA Run:ai brings together major open source projects and integrated platform capabilities for operating AI infrastructure at scale. Deploy the complete platform, adopt a specific project, or combine NVIDIA Run:ai with your existing software, infrastructure, and services.

NVIDIA Run:ai Is Fully Open Source

Organizations can deploy the integrated platform, adopt individual projects, extend NVIDIA Run:ai, and contribute to its development.

Explore NVIDIA Run:ai on GitHub

Platform

AI Workload and GPU Orchestration to Build, Train, and Deploy AI Workloads at Scale

AI-Native Workload Orchestration

Centralize and automate training, fine-tuning, and inference across teams, clusters, and distributed GPU environments.

Dynamic GPU Allocation

Keep GPUs productive by dynamically matching resources to workload demand and reducing idle or fragmented capacity.

Policy-Driven Governance

Manage quotas, priorities, access controls, and workload fairness across departments, projects, and teams, so critical workloads receive the right resources.

Open Source Foundation

Adopt the complete platform, use specific projects, or integrate NVIDIA Run:ai with existing control planes, user experiences, infrastructure services, and operational tooling.

Performance

Real-World AI Acceleration: Proven GPU Orchestration at Scale

Dynamic scheduling and orchestration that accelerates AI throughput, delivers seamless scaling, and maximizes GPU utilization.

10x

GPU Availability

20x

Workloads Running

GPU Utilization

Manual Intervention

Open Source

Build With Open Source NVIDIA Run:ai Projects

Schedule AI Workloads at Scale With KAI Scheduler

KAI Scheduler is an open source Kubernetes scheduler designed for AI workloads and shared GPU infrastructure. Use it when building a Kubernetes-based AI platform, cloud service, or shared cluster that requires specialized AI scheduling, hierarchical queues, quotas, and fair-share, priority, preemption, and topology-aware placement.

Explore KAI Scheduler on GitHub

Apply Fractional GPU Allocations With KAI GPU Fractioning Operator

KAI GPU Sharing Operator translates GPU-sharing decisions into enforceable runtime controls for Kubernetes workloads. Use it when building a platform that needs flexible GPU sharing, workload-level boundaries, and per-fraction resource telemetry.

Explore KAI GPU Fractioning Operator on GitHub

Accelerate Model Loading With NVIDIA Run:ai Model Streamer

NVIDIA Run:ai Model Streamer is an open source Python SDK for accelerating model loading by streaming tensors concurrently from storage toward GPU memory. Use it in inference runtimes, serving platforms, or autoscaling systems where model-loading latency delays readiness and GPU productivity.

Explore Model Streamer on GitHub

Benefits

Scale AI Workloads and Make Every GPU Count

Purpose-built for AI workloads and shared GPU infrastructure, NVIDIA Run:ai helps developers move faster, platform teams govern resources, and organizations improve AI infrastructure economics.

Maximize GPU Utilization and Reduce Waste

Dynamically pool, schedule, and allocate GPU resources based on workload demand and business priority. Reduce idle and fragmented capacity, increase throughput, and improve the return on GPU infrastructure investments.

Accelerate AI From Development to Production

Give developers on-demand access to shared GPU resources for training, fine-tuning, and inference. Intelligent scheduling reduces waiting and infrastructure friction, so teams can iterate faster and move AI workloads into production reliably.

Leverage Centralized Governance for Shared AI Infrastructure

Give platform teams visibility and control across workloads, users, clusters, and environments. Policy-based quotas, priorities, access controls, and workload fairness help critical AI workloads receive the right resources while teams share infrastructure efficiently.

Integrate Across Any Environment

Deploy NVIDIA Run:ai across public cloud, private cloud, hybrid, and on-premises environments. Its open source foundation and API-first architecture allow teams and partners to integrate NVIDIA Run:ai with existing AI frameworks, infrastructure, operational tooling, and differentiated user experiences.

Use Cases

Accelerating AI Workloads With Intelligent Orchestration

Purpose-built for AI workloads, NVIDIA Run:ai delivers intelligent orchestration that maximizes compute efficiency and dynamically scales AI training and inference.

Survey Report

State of AI in Telecommunications

The 2026 survey explores how telecom companies are investing in, deploying, and benefiting from AI.

Download Now

Scaled AI
Fractional Inference
Mitigating Cold Start

Enterprise AI Acceleration

NVIDIA Run:ai enables enterprises to scale AI workloads efficiently, reducing costs and improving AI development cycles. By dynamically allocating GPU resources, organizations can maximize compute utilization, reduce idle time, and accelerate machine learning initiatives. NVIDIA Run:ai also simplifies AI operations by providing a unified management interface, enabling seamless collaboration between data scientists, engineers, and IT teams.

Watch Video (08:02)

Need Help Selecting the Right Partner?

Visit the NVIDIA Partner Network Locator to find your preferred NVIDIA partners certified to provide NVIDIA Run:ai.

Learn More

Maximizing Token Throughput

Run diverse AI workloads concurrently on shared GPU infrastructure to dramatically increase total throughput and utilization. By fractionally allocating GPUs across inference, embedding, and generation tasks, organizations can run more models in parallel without resource contention. Compared to single-model, full-GPU execution, mixed workloads deliver significantly higher aggregate throughput at the GPU, host, and cluster level—maximizing infrastructure efficiency while accelerating AI output across teams.

Read Blog

Mitigating Model Cold Start

Reduce model deployment costs without sacrificing performance by dynamically swapping model memory between GPU and host. NVIDIA’s GPU memory swap approach keeps active parts of the model resident on GPU while transparently paging inactive portions, enabling larger models to run on fewer GPUs. This reduces infrastructure spend, lowers idle capacity, and supports cost-efficient inference for production deployments—especially for memory-intensive large language model workloads.

Read Blog

How to Get NVIDIA Run:ai

In addition to open source, NVIDIA Run:ai is available as supported software through NVIDIA DSX OS™, NVIDIA Mission Control™, and NVIDIA AI Enterprise or as standalone software. Select the option that best fits your infrastructure and operating model.

NVIDIA DSX OS

Build and operate leading AI cloud factories with NVIDIA DSX OS, a portfolio of open source, modular, and composable-by-design software that helps partners build and operate AI factories at scale reliably, efficiently, and securely.

Learn More

NVIDIA Mission Control

Streamline AI factory operations—from workload scheduling and orchestration to infrastructure monitoring and autonomous recovery—with Mission Control, which includes NVIDIA Run:ai technology.

Learn More

NVIDIA AI Enterprise

Access NVIDIA Run:ai as part of NVIDIA AI Enterprise, a fully supported, production-ready commercial software suite for developing and deploying AI at scale.

Learn More

NVIDIA Run:ai Standalone

Deploy the complete NVIDIA Run:ai platform as standalone software with NVIDIA enterprise support and lifecycle services for production environments.

Contact NVIDIA

Ecosystem

NVIDIA Run:ai Across the AI Ecosystem

A growing ecosystem of cloud providers, infrastructure vendors, software companies, systems integrators, and managed service providers is building with and delivering NVIDIA Run:ai. Find NVIDIA Run:ai through participating CSP marketplaces or work with a partner for integration, implementation, support, or managed services