White Paper
AI Tokenomics: A Framework for Deploying and Monetizing Inference at Scale
Introduction to Token Economics
AI runs on tokens. How you use, scale, generate, and monetize them determines your AI competitive advantage. Select the optimal model and context length for each use case, deploy infrastructure that maximizes tokens per watt and minimizes cost per token, and drive revenue and margin growth at scale.
Data centers are becoming AI token factories. Where they once processed transactions and served webpages, they now manufacture a new kind of output: tokens, the unit of intelligence produced by AI models during inference. As AI moves from experimentation into production, inference has become the dominant workload, and tokens have become the new commodity that enterprises must learn to produce, manage, and monetize.
What is tokenomics? It is the economics of token generation and usage and is a term frequently used by the ecosystem, but it can lack precision. To drill down into what tokenomics really means, it’s helpful to frame it as how tokens are valued, consumed, supplied, and monetized.
Tokenomics encompasses four interconnected pillars:
- Token Utility: Describes how a token is valued and how the required value varies by use case
- Token Demand: Covers how to forecast the volume of tokens a deployment will require, accounting for workload characteristics and operational conditions
- Token Supply: Addresses how to maximize token availability while minimizing the cost to produce them
- Token Monetization: Explores how to turn token output into revenue and sustainable margins
Each pillar of tokenomics doesn’t exist in isolation. Decisions about supply directly affect margins, demand forecasts determine infrastructure requirements, and the token utility sets the ceiling on what can be charged. These four pillars are deeply interdependent, and getting them right is what separates sustainable AI businesses from costly experiments. This paper explores each in depth, providing a framework for enterprises planning or scaling AI deployments.
Token Utility: Understanding Value and Use Cases
Not all tokens are created equal. The value of a token depends on two dimensions:
- Intelligence scales with model size and complexity, as well as with the amount of context the model processes. A token produced by a frontier reasoning model with a 400,000-token context window carries more intelligence than one produced by a lightweight model with a 1,000-token context window.
- Interactivity is the speed at which tokens are delivered. A token that arrives in near-real time is worth more to an interactive application than one that arrives seconds later.
The higher the intelligence and the faster the interactivity, the costlier those tokens are to produce. But the value captured from these tokens is relative to the use case and depends entirely on whether the use case can take advantage of them. A post-trained small language model (SLM) can match or exceed a much larger, more intelligent model for domain-specific tasks at a fraction of the cost. A batch document-processing pipeline has very different interactivity requirements from a real-time coding assistant. The additional value is only worth paying for if the application can use it.
Token utility is best understood as a spectrum: from high-throughput, lower-interact ivity workloads served by basic or post-trained SLMs, such as search and chatbot, to near-real-time workloads requiring the largest models with the longest context, such as agentic coding and automation. The strategic task is mapping each use case to the right point on that spectrum.
A practical approach is to prototype and validate with frontier capability first, then define strict service-level objectives (SLOs) through evaluation, and finally optimize and right-size for scale. Questions that help guide that process include:
- Model Archetype: Does the task require versatility across topics or specificity to a narrow domain?
- Intelligence Mode: Does the task require reasoning or retrieval-augmented generation (RAG)?
- Intelligence and Cost: What’s the model accuracy versus cost tradeoff?
- Value Persistence: How long does the answer stay useful, and does that justify a higher inference cost?
- Risk Tolerance: What is the cost of an inaccurate response?
- Execution Profile: Is the task better served by autonomous agency or a predictable, structured flow?
- Tokens per task: For agentic AI, what is the turns per task, and tokens per turn?
Getting this mapping right has direct downstream consequences. It determines which models are deployed, how to size infrastructure needs, and ultimately what to charge for the tokens produced. Token utility is the foundation on which demand, supply, and monetization decisions are built.
NVIDIA AI infrastructure provides customers the flexibility to choose optimal token value by supporting a broad range of proprietary and open-weight models. NVIDIA also builds Nemotron, a family of highly efficient, multimodal, open models built for long-running, self-evolving agents. For example, Nemotron 3 Ultra completed coding benchmarks using fewer total tokens and fewer tokens per turn than comparable models, lowering the cost for agentic tasks by up to 30%.
Token Demand: A Three-Level Forecasting Model
How do you forecast token demand? Starting with the use case is the most effective way to use demand as a forecasting and-infrastructure-planning tool. The market is experiencing an aggressive expansion of use cases, from consumer and enterprise chatbots to coding and agentic automation. This shift is quickly transforming the demand landscape from a market of trillions of tokens to quadrillions, fundamentally changing the workload and the demand characteristics of tokens and their unit cost.
Businesses require methodologies to forecast their own demand for planning capital and operational expenditures associated with their AI use cases. The first-principles driven approach consists of three levels:
- Level 1, Base-Demand Estimates: Ballpark estimates based on user volume and per-request size metrics
- Level 2, Workload-Demand Multipliers: Introduces scaling parameters based on the nature of the workload and how it’s consumed by end users
- Level 3, Operational Demand: Requires a deep understanding of the deployment environment and advanced telemetry
A three-layer token-demand forecasting model, from base estimates through workload multipliers to operational demand variables that shape real-world infrastructure requirements.The three-level model with color coding: Green indicates shared components with traditional software capacity planning, and orange highlights variables unique to token serving (tokens per request, reasoning, agent loops).
The demand estimates based on level 1 are where capacity planning starts. They serve as a baseline understanding of requirements for proof-of-concept deployments, but they rarely scale to production. Workload-demand multipliers can scale token demand by an order of magnitude. Shown in the example below, if infrastructure was provisioned based solely on the base demand estimate, the associated application would be short over 600 million tokens per day.
The operational demand (layer 3) considerations require modeling demand at a more granular level. This token-demand forecasting worksheet provides a fill-in-the-blank version to walk you through all three levels.
Token-demand forecasting is less of a departure from traditional software capacity planning than it might appear. Most inputs—sessions, requests, retries, cache hit rates, temporal patterns, and latency service-level agreements (SLAs)—are shared with practices that infrastructure teams already know. Three variables are unique to serving tokens: tokens per request, reasoning overhead, and agent loops. What differentiates one deployment's demand profile from another's isn’t methodology but context: the use case, the organization deploying it, and the application design decisions baked into the workflow.
Organizations that treat these three levels as a living model rather than a one-time exercise will be best positioned to right-size infrastructure as agentic workloads scale and token demand compounds.
Token Supply: How to Maximize AI Tokens, Minimize Cost
Token supply is fundamentally about efficiency: maximizing token output while minimizing the cost of producing tokens.
Achieving this requires co-optimizing across model efficiency, system efficiency, and software efficiency. No single component is sufficient on its own. All three must be designed together with the ecosystem. NVIDIA calls this “extreme co-design,” and it spans models and algorithms, compute, networking, memory, storage, software, and the ecosystem of partners and customers, with each component working together to make the whole greater than the sum of its parts and enables the system to generate the lowest token cost.
Model Efficiency
Mixture-of-experts (MoE) architectures deliver significant gains in model efficiency by activating only the relevant parameters for every token. For example, Kimi K2.5 has 1 trillion parameters but activates only 32 billion per token, delivering the intelligence of a much larger dense model at a fraction of the compute cost. The hidden cost is communication between experts spread across GPUs. MoE inference generates heavy all-to-all traffic between GPUs across batches, and efficiency gains evaporate if the underlying system can’t handle that traffic at scale.
System Efficiency
The NVIDIA GB200 NVL72 connects 72 GPUs through the NVIDIA NVLink™ Switch, enabling 1,800 gigabytes per second (GB/s) of all-to-all bandwidth. This larger GPU domain allows experts to be distributed across up to 72 GPUs without the communication bottlenecks that constrain fixed-mesh, single-node configurations or off-the-shelf Ethernet-based scale-up approaches. This ensures that the theoretical efficiency of MoE models is realized in production.
Software Efficiency
Realizing the full potential of the hardware requires a software stack optimized across three architectural layers: infrastructure access, application acceleration, and orchestration and serving. NVIDIA delivers a robust software stack that can enable every optimization simultaneously. This is harder than it sounds, but the compounding effect is what produces step-change improvements in delivered token output. Continuous software optimizations by the open source ecosystem and NVIDIA make existing hardware more productive over time.
The result of extreme co-design is the lowest cost per token in the industry on GB200 NVL72.
Optimizing Within Enterprise Constraints
Most enterprises need to optimize token cost within existing data center footprints and environment constraints. For instance, the average rack power density is around 27 kilowatts (kW), and 75 percent of data centers are still air-cooled instead of water-cooled. Enterprises also often have mixed workloads outside of AI, such as data or graphic processing, and limited upfront capital availability to address these constraints. NVIDIA offers purpose-fit options. The NVIDIA HGX™ B200 and B300 deliver the lowest token cost for air-cooled AI factories. The NVIDIA RTX PRO™ 6000 Blackwell delivers up to 3x the token efficiency of prior-generation NVIDIA Hopper™ systems for enterprise inference workloads.
Token Generation Efficiency
Throughput per Megawatt and Cost per Token: The Metrics That Matter
Most organizations still evaluate AI infrastructure using input metrics such as cost per GPU-hour, peak floating-point operations per second (FLOPS), or FLOPS per dollar. These are the wrong metrics for evaluating AI infrastructure. Four measurements comparing NVIDIA Blackwell and Hopper makes the case:
- Compute Cost: 2x higher
- FLOPS per Dollar: 2x higher
- Cost per Million Tokens: 35x lower
- Token Output per Second per MW: 50x higher
The metrics that matter most are throughput per MW and cost per token.
Throughput per MW impacts your revenue and cost per token determines profitability per interaction and how fast a business can grow. These metrics capture hardware performance, software optimization, and real-world utilization in a single metric. For a deeper treatment of inference total cost of ownership, see Rethinking AI TCO: Why Cost per Token Is the Only Metric That Matters.
As agentic AI matures, adjacent ideas are emerging alongside cost per token: intelligence per token, tokens per task, and cost per task. Cost per task is a function of how efficiently a model generates intelligence (intelligence per token) and how efficiently a model uses tokens to complete a task (tokens per task).
Token Monetization: How to Price AI Tokens
Utility defines what a token is worth. Demand forecasts how many tokens will be consumed and by whom. Supply decides what it costs to produce.
Monetization brings these three pillars together and answers the central business question: How do you price and sell tokens so that the economics are favorable and sustainable?
There’s no single economic model for how to monetize tokens. Four key strategies have emerged, each suited to different business contexts and starting points.
The pricing mechanics that follow are based on selling tokens directly. For products and services built on top of tokens, the approach is conceptually similar, though organizations must also account for the additional value they’re delivering to the end customer.
Getting the pricing fundamentals right is foundational regardless of which model an organization pursues.
How to Determine Token Pricing
When pricing tokens, consider three important factors:
- Cost-Based Pricing: The cost to produce a token sets the floor. Pricing below it means selling at a loss.
- Value-Based Pricing: The value a token delivers to the buyer sets the ceiling. Enterprises need to understand the value of tokens to understand a customer’s willingness to pay for those tokens. This lets enterprises price tokens accordingly to maximize business and customer value. To note, token value isn’t uniform: Tokens carrying higher intelligence and interactivity command higher prices.
- Revenue, Profit, and Demand Distribution: As price rises, overall demand contracts. Pricing strategy has to account for how demand shifts across the token utility tiers and whether the resulting revenue and margin meet business goals.
Ultimately, token revenue depends on token output, token price, and the distribution of demand across pricing tiers. Profit margins depend on the cost to produce tokens. Infrastructure decisions are thus directly relevant to revenue. The NVIDIA Vera Rubin platform boosts throughput and lowers cost per token by 10x. Additionally, Vera Rubin with LPX unlocks high-interactivity, high-intelligence workloads that command premium prices, expanding both revenue opportunities and margins at the top of the pricing spectrum.
From Tokens to Business Value: Case Studies Across the Four AI Business Models
The following case studies illustrate how organizations are putting these strategies into practice across each of the four models: selling tokens directly, building AI-native offerings, enhancing existing products and services, and transforming internal operations. The performance and efficiency numbers presented earlier in this guide highlight what's achievable with full co-design. The case studies that follow show organizations at various stages of that journey, where results reflect their specific combination of model selection, sequence lengths, numerical precision, inference optimizations, and deployed hardware.
-
Cohere
-
Perplexity
-
Canva
Case Study: AI Native
Summary
Cohere, founded in 2019, is a leading sovereign AI company building foundation models and end-to-end products to solve real-world business problems. Cohere works on both sides of the token economy: They build their own models and they build the products and solutions that use those models. Cohere’s primary family of large language models (LLMs) is the Command/North family of models, including Command A Vision (multimodal) and Command A Translate, plus Embed and Rerank for search.
Cohere also developed North, a secure enterprise AI workspace where employees work alongside customizable AI agents that search company data with RAG, use tools, run workflows, and power automations.
Goal
As North runs on Cohere’s own models, every interaction is token hungry and latency sensitive, so inference efficiency directly drives Cohere’s margins and product quality. Because of this, Cohere needs the best AI infrastructure to provide the lowest token cost, reduce latency, and achieve high throughput per node. Cohere runs a continuous benchmarking program across models, hardware, and load levels.
Solution
Cohere moved its production models from NVIDIA H100 to NVIDIA Blackwell B200 and GB200, typically running at FP8 precision.
Results
In Cohere’s own internal benchmarks comparing NVIDIA H100 to NVIDIA B200, the team found:
- 1.6–1.9x more tokens per second per node were produced on Command A+ and Command A Vision.
- Up to 2.6x more tokens per second per node were produced on Command A Translate.
- 30–50 percent lower time to first token was achieved on most mid- and long-context workloads.
- Gains compound under load. On matched, clean, high-concurrency points, NVIDIA Blackwell sustains about 1.7x or more the throughput of NVIDIA Hopper.
Bar graph showing B200 matched-sweep peak throughput gain over H100 (up to 64 users).
Technical Gains Lead to Business Value
Higher throughput per node and lower latency convert into margins and a better product.
Overall, Cohere has achieved better margins, greater scale, greater flexibility, and a better customer experience by delivering its models and solutions on NVIDIA Blackwell.
- Margins: Lower cost per token on high-volume workloads widens gross margin on every North seat and API call.
- Scale: 1.6–2.6x more throughput per node serves more users and agents before adding hardware.
- Flexibility: The same NVIDIA stack runs on premises within customers’ virtual private clouds, maintaining healthy unit economics for regulated enterprises.
- Customer Experience: 30–50 percent faster time to first token makes North’s agents remain performant and fast, even when dealing with large document sets.
To illustrate, the table below shows the general business impact Cohere has achieved with NVIDIA Blackwell.
Note: The measured throughput is combined with publicly available GPU prices to illustrate the shape of the impact, not Cohere’s actual costs.
Illustrative cost per 1 million tokens (same GPU count; illustrative $3.50/H100-hr and $6.00/B200-hr = 1.71x price premium).
| Model (Scenario) | H100 Cost per 1M Tokens | B200 Cost per 1M Tokens | Throughput Gain | Tokens per Dollar |
|---|---|---|---|---|
| Command A Translate (1K/100) | $0.39 | $0.26 | 2.62x | +53% (-35% cost) |
| Command A+ (10K/1K) | $0.128 | $0.113 | 1.93x | +13% (-11% cost) |
| Command A Vision (1 image) | $1.99 | $1.83 | 1.87x | +9% (-8% cost) |
Tokens-per-dollar gain = throughput gain / GPU price premium. Above ~1.7x throughput (long-context, high-concurrency, translation) NVIDIA Blackwell yields real cost-per-token savings. The advantage grows as the B200/H100 price gap narrows (cloud rates are often closer to 1.5x).
Case Study: AI Native
Summary
Perplexity is an AI agent platform focused on accurate AI. The company’s platform leverages 15 AI models, including NVIDIA Nemotron™ 3 Ultra, routing each workload through the pipeline to the model with the best cost-quality fit to optimize performance, quality, and cost for our customers.
"Every generation of NVIDIA hardware has bought us something specific. NVIDIA Hopper let us serve our model fleet. NVIDIA Blackwell, with the larger NVLink domain and MXFP8 on the Tensor Cores, lets us serve it across multiple nodes without paying for it in latency. That's what makes the workloads we're moving into economically viable." – Denis Yarats, Co-Founder and CTO, Perplexity
Goal
Perplexity’s business goal is to deliver accurate AI at consumer scale while preserving the economics required for enterprise-grade reliability and growth. Perplexity's production infrastructure already serves more than 50 million queries a day across a 15 model fleet, at a 50th percentile time to first token (P50 TTFT), ranging from 2.4 seconds on fast search to ~13 seconds on deep reasoning work. As more of that traffic shifts into multi-hour agentic work in Perplexity Computer, Perplexity makes each unit of intelligence cheaper, faster, and more reliable, so cost and latency scale sublinearly with traffic and task complexity.
Solution
Perplexity’s AI infrastructure is built on multiple generations of NVIDIA hardware, including NVIDIA Hopper and NVIDIA Blackwell, and is orchestrated and optimized with NVIDIA’s data center software stack.
Perplexity built its inference stack on NVIDIA to drive down cost per token without giving up accuracy or latency. Each successive GPU generation performance improves significantly and, when paired with FlashInfer, CuTe DSL, SHARP, MXFP8 quantization, and disaggregated prefill and decode serving, they’re able to extract more tokens per GPU-hour.
Results
Perplexity achieved an approximately 2-4x improvement in output token cost on multi-node GB200 versus single-node H200, while also reducing dispatch latency by 46.5% on MoE models.
Reduced Latency
The chart below compares MoE dispatch latency across H200 and GB200 using NVLink and NVLS transports, showing GB200 with NVLS cuts dispatch latency by 46.5 percent versus H200 with NVLS (313.3 μs vs. 586.1 μs).
Perplexity
Improved Throughput
The next chart plots per-GPU decode throughput against decoding speed for GB200 (EP = 4/8/16) and H200 (EP = 8/16), showing GB200 multi-node deployments achieve 2–4x improved throughput per GPU compared to H200 single-node at matched decoding speeds, thanks to the larger NVLink domain.
Perplexity
Case Study: Product Enhancement
Summary
Canva’s mission is to empower the world to design. Canva’s suite of AI design tools can help anyone do their best work, from social media posts to presentations and short videos. Among the most popular features of their Design suite is the image-to-video AI generator, which transforms a still photo into a short animated clip, requiring no software and working on any device.
"Working with NVIDIA helps Canva improve the efficiency of AI-powered creative experiences like image to video while preserving the speed and quality our users expect. That efficiency is key to making advanced creative tools accessible to more people." – Stefano Corazza, Head of AI Research, Canva
Goal
Canva currently has over 265 million monthly active users, making them one of the three most widely used AI applications in the world. To make the popular image-to-video AI generator available to everyone, Canva has built out specialized inference infrastructure powered by NVIDIA. This ensures the generated videos meet the highest industry standards while keeping cost per generation and latency at reasonable levels.
Solution
Canva runs their image-to-video inference stack on NVIDIA to reduce cost per generation without sacrificing accuracy or latency. The feature is powered by a diffusion-transformer video model, served on NVIDIA B200s. Thanks to optimized distributed inference and post-processing pipelines combined with hardware-specialized kernels, Canva delivers more generated video per GPU-hour, allowing the free image-to-video experience to scale to a global user base.
Results
Canva was able to achieve 70 percent more video generations per GPU-hour on B200s versus H200s.
Token-Demand Forecasting Worksheet
How to Use This Worksheet: Fill in each [_blank_] with the appropriate number. Each layer's outputs feed into the next layer. The math under every section shows exactly what's being multiplied and how the numbers cascade.
Layer 1: Base Demand
We're sizing this deployment for [__________] users, each running about [__________] tasks every day. A typical request sends [__________] input tokens, and the model gives back [__________] output tokens.
Inputs
A. Users ......................... [__________]
B. Tasks per user per day ........ [__________]
C. Input sequence length (ISL) ...... [__________] tokens<
D. Output sequence length (OSL) ...... [__________] tokens
The Math
Sessions / Day = A × B
= [_____] × [_____]
= [_______________] sessions/day
Tokens / Request = C + D
= [_____] + [_____]
= [_______________] tokens/request
Base Tokens / Day = Sessions × Tokens/Request
= [_____] × [_____]
= [_______________] tokens/day
Base Tokens / Month = Daily × 30
= [_______________] tokens/month
Layer 2: Workload Multipliers
Each step burns about [__________] hidden reasoning tokens, and a single task chains through [_____] agent steps. Failures and fallbacks add a [_____]× retry overhead, and we attach a [__________] token system prompt to every call. Prompt caching is hitting [_____]% of system-prompt tokens.
Inputs
E. Reasoning tokens per step ....... [__________]
F. Agent steps per task ............ [_____]
G. Retry multiplier ................ [_____] ×
H. System prompt tokens per step ... [__________]
I. Cache hit rate .................. [_____] %
The Math
Adjusted Tokens / Request = Tokens/Req + E
= [_____] + [_____]
= [_______________]
Tokens / Task (with steps) = Adjusted × F
= [_____] × [_____]
= [_______________]
Tokens / Task (with retries) = Above × G
= [_____] × [_____]
= [_______________]
Cache Savings / Day = H × (I / 100) × F × Sessions
= [_____] × [_____] × [_____] × [_____]
= [_______________] tokens saved/day
Effective Tokens / Day = (Tokens/Task × Sessions) − Cache Savings
= [_____] − [_____]
= [_______________] tokens/day
Effective Tokens / Month = Daily × 30
= [_______________] tokens/month
Layer 3: Operational Shaping
About [_____]% of daily traffic lands inside a [_____]-hour business window. Inside that peak, real traffic bursts to [_____]× the windowed average, and seasonal spikes stretch us another [_____]×. To meet latency SLAs, maintain [_____]× extra headroom.
Inputs
J. Peak hour concentration ......... [_____] %
K. Peak window hours ............... [_____] hours
L. Burst ratio ..................... [_____] ×
M. Seasonal spike factor ........... [_____] ×
N. Latency SLA headroom ............ [_____] ×
The Math
Avg TPM (24-hour) = Effective Daily ÷ 1,440
= [_____] ÷ 1,440
= [_______________] tokens/min
Avg TPM (peak window) = (Effective Daily × J/100) ÷ (K × 60)
= ([_____] × [_____]) ÷ ([_____] × 60)
= [_______________] tokens/min
Peak Burst TPM = Peak TPM × L
= [_____] × [_____]
= [_______________] tokens/min
Peak TPM + SLA = Burst × N
= [_____] × [_____]
= [_______________] tokens/min
Peak TPM + Seasonal = Above × M
= [_____] × [_____]
= [_______________] tokens/min
Get Connected with Sales
NVIDIA tokenomics focuses on optimizing the cost and revenue of generating AI tokens per watt, aiming to lower the overall cost per token through architectures like NVIDIA Blackwell and Vera Rubin. To discuss infrastructure pricing or software licensing, you can reach out via the form below.