NVIDIA NIM Microservices
Designed for rapid, reliable deployment of accelerated AI models anywhere.
Overview
What Is NVIDIA NIM?
NVIDIA NIM™ provides prebuilt, optimized inference microservices for rapidly deploying the latest AI models on any NVIDIA-accelerated infrastructure—cloud, data center, workstation, and edge.
Accelerate AI Deployment With NVIDIA NIM
NVIDIA NIM combines the ease of use and operational simplicity of managed APIs with the flexibility and security of self-hosting models on your preferred infrastructure. NIM microservices come with everything AI teams need—the latest AI foundation models, optimized inference engines, industry-standard APIs, and runtime dependencies—prepackaged in enterprise-grade software containers ready to deploy and scale anywhere.
Benefits
Deploy AI Models Faster—With Performance Built In
Move from prototype to production with prebuilt, optimized inference microservices for the latest AI models. NVIDIA NIM delivers low-latency, high-throughput inference through industry-standard APIs, with the flexibility to deploy across cloud, data center, workstation, and edge.
Ease of Use
Accelerate innovation and time to market with prebuilt, optimized microservices for the latest AI models. With standard APIs, models can be deployed in five minutes and easily integrated into applications.
Enterprise Grade
Deploy enterprise-grade microservices that are continuously managed by NVIDIA through rigorous validation processes and dedicated feature branches—all backed by NVIDIA enterprise support, which also offers direct access to NVIDIA AI experts.
Performance and Scale
Improve TCO with low-latency, high-throughput AI inference that scales with the cloud, and achieve the best accuracy with support for fine-tuned models out of the box.
Portability
Deploy anywhere with prebuilt, cloud-native microservices ready to run on any NVIDIA-accelerated infrastructure—cloud, data center, and workstation—and scale seamlessly on Kubernetes and cloud service provider environments.
Demo
Build AI Agents With NIM
Learn how to set up two AI agents—one for content generation and another for digital graphic design—and see how easy it is to get up and running with NIM microservices.
Technology
Building Blocks for Agentic AI
Benchmarks
Boost Throughput With NIM
NVIDIA NIM provides optimized throughput and latency out of the box to maximize token generation, support concurrent users at peak times, and improve responsiveness. NIM microservices are continuously updated with the latest optimized inference engines, boosting performance on the same infrastructure over time.
Configuration: Llama 3.1 8B instruct, 1x H100 SXM; concurrent requests: 200. NIM ON: FP8, throughput 1201 tokens/s, ITL 32ms. NIM OFF: FP8, throughput 613 tokens/sec, ITL 37ms.
Models
Unlock Enterprise-Ready Inference for Thousands of Open Models
Deploy LLMs supported by NVIDIA TensorRT™-LLM, vLLM, or SGLang for low-latency, high-throughput inferencing on NVIDIA-accelerated infrastructure.
Features
The Easy Button for AI Development and Deployment
Designed to run anywhere, NIM microservices expose industry-standard APIs for easy integration with enterprise systems and applications and scale seamlessly on Kubernetes to deliver high-throughput, low-latency inference at cloud scale.
Deploy NIM
Deploy NIM for your model with a single command. You can also easily run NIM with LLMs supported by NVIDIA TensorRT-LLM, vLLM, or SGLang, including fine-tuned models.
Run Inference
Get NIM up and running with the optimal runtime engine based on your NVIDIA-accelerated infrastructure.
Build
Integrate self-hosted NIM endpoints with just a few lines of code.
Use Cases
How NIM Is Being Used
See how NVIDIA NIM supports industry use cases, and jump-start your AI development with curated examples.
-
AI Assistants
-
Document Intelligence
-
Hyperpersonalized Shopping
-
3D Product Configurators
AI Assistants
Enhance customer experiences and improve business processes with generative AI.
Intelligent Document Processing
Use generative AI to accelerate and automate document processing.
AI for Hyperpersonalized Shopping
Deliver tailored experiences that enhance customer satisfaction with the power of AI.
3D Product Configurators
Use OpenUSD and generative AI to develop and deploy 3D product configurator tools and experiences to nearly any device.
Starting Options
Ways to Get Started With NVIDIA NIM
Resources
The Latest NVIDIA NIM Resources
Next Steps
