Inside AI Tokenomics: How to Profitably Turn Tokens Into Business Value | NVIDIA AI Podcast Ep. 299

In short

In this episode of the Nvidia AI Podcast, Noah Kravitz interviews Shruti Koparkar about tokenomics in AI. They define tokenomics as the valuation, supply, demand, and monetization of tokens, emphasizing the importance of aligning AI infrastructure decisions with business goals. The discussion covers token utility, the impact of model intelligence and interactivity on token value, and strategies for optimizing token supply and demand. They also discuss extreme co-design and the critical role of software in maximizing token throughput and reducing costs. The episode concludes with practical advice for business leaders on implementing tokenomics in their organizations.

Key takeaways

• Tokenomics is about valuing, supplying, consuming, and monetizing tokens to maximize business value.
• Token value depends on the intelligence embedded in the token and its interactivity, which should align with the specific use case.
• AI infrastructure decisions should focus on cost per token to optimize token output while minimizing costs.
• Extreme co-design, involving simultaneous design of hardware and software, is crucial for achieving the lowest token cost.
• Start with the customer and use case to drive token utility, demand, supply, and monetization strategies.

Chapters

1. 00:00

   Introduction to Tokenomics

   Noah Kravitz introduces Shruti Koparkar from Nvidia's accelerated computing team to discuss tokenomics in the context of AI data centers becoming AI factories.

2. 01:00

   Defining Tokenomics

   Shruti defines tokenomics as how tokens are valued, supplied, consumed, and monetized, mapping to token utility, supply, demand, and monetization.

3. 02:02

   Token Utility and Value

   Shruti explains that not all tokens are created equal, and their value depends on the intelligence embedded in the token and how fast it arrives (interactivity).

4. 03:39

   Defining Token Value

   The value of a token is tied to the task at hand. More complex models and longer contexts generally produce more valuable tokens, but the specific use case dictates the required level of intelligence and interactivity.

5. 06:50

   Token Demand and Use Cases

   Shruti discusses how to think through token demand by considering the number of users, requests, and tokens needed per request, along with multipliers like reasoning models, agentic workflows, and cache hit rates.

6. 11:50

   Token Supply and AI Infrastructure

   Token supply involves AI infrastructure decisions aimed at maximizing token output while minimizing costs, focusing on metrics like cost per token rather than just input metrics like cost per GPU hour.

7. 16:53

   Extreme Co-design

   Shruti explains extreme co-design as designing multiple parts of a system simultaneously from the ground up, optimizing for the lowest token cost across compute, memory, storage, networking, and software.

8. 24:40

   The Role of Software

   Software is crucial for realizing the full potential of hardware, enabling optimizations like NVFP4 quantization, speculative decoding, disaggregated serving, and wide expert parallel to maximize token throughput and reduce costs.

9. 27:37

   Monetizing Tokens

   Shruti outlines business models for monetizing tokens, including selling tokens directly, building AI-native products, enhancing existing products with AI, and improving internal operations with AI.

10. 31:50

    Putting Tokenomics into Practice

    Shruti advises starting with the customer and use case to determine the required model, context length, and interactivity, then making infrastructure decisions based on cost per token and developing a monetization strategy.

Get the next The AI Podcast recap

A short recap of every new episode, by email. Free for up to 5 shows.