# Scaling AI Agents Globally Across Brands, Use Cases, and Restaurants

## Abstract

Scaling and deploying AI into restaurant operations requires unique technology and partners. Learn how Yum! Brands, the world’s largest restaurant company, is using NVIDIA software to scale intelligence into their restaurant technology, expanding use cases, applications, and partners.

## AI Summary

- The speaker emphasized the importance of governance in AI, stating that intelligence without governance can lead to catastrophic failures.
- Yum Brands transitioned from proof of concepts to industrializing AI, focusing on scalable skills and specific capabilities rather than monolithic use cases.
- The company developed Byte, a proprietary e-commerce platform, to integrate AI across various systems and create an AI flywheel, improving experiences and gathering more outcome data.
- Yum Brands adopted open-source small language models (SLMs) and used synthetic data generation to train models, achieving significant improvements in accuracy and cost reduction.
- Tool call correctness was identified as a critical product in the development process, ensuring that AI models interacted accurately with different systems and workflows.
- The partnership with NVIDIA and the use of NeMo tools were crucial in enabling Yum Brands to build, train, and deploy AI models efficiently across their global operations.

## Transcript

**[00:00:09 – 00:00:11]** Good afternoon everyone.

**[00:00:11 – 00:00:13]** Welcome to GTC.

**[00:00:13 – 00:00:14]** My name is Andrew Sun.

**[00:00:14 – 00:00:16]** I help lead our global business development for the retail

**[00:00:16 – 00:00:20]** industry, which is retail, consumer goods, and quick

**[00:00:20 – 00:00:21]** service restaurants.

**[00:00:21 – 00:00:25]** And I'm excited for our panel today and to welcome our guests

**[00:00:25 – 00:00:28]** here, Cam and Joey.

**[00:00:28 – 00:00:31]** Today, when you look at these industries, it's over a trillion

**[00:00:31 – 00:00:32]** dollars in commerce.

**[00:00:32 – 00:00:36]** And not only are these big industries, but these are

**[00:00:36 – 00:00:38]** industries that involve a lot of people and a lot of

**[00:00:38 – 00:00:39]** volume and a lot of scale.

**[00:00:40 – 00:00:43]** And so talking about agents is really fundamental to how

**[00:00:43 – 00:00:45]** this industry is going to evolve.

**[00:00:45 – 00:00:48]** We're really excited today because we're welcoming Young

**[00:00:48 – 00:00:51]** Brands, which is the world's largest enterprise when it

**[00:00:51 – 00:00:53]** comes to the restaurant business.

**[00:00:53 – 00:00:57]** And you're going to hear about their journey from AI proof of

**[00:00:57 – 00:01:02]** concepts into industrializing AI, producing intelligence and making

**[00:01:02 – 00:01:05]** it scale across their business.

**[00:01:05 – 00:01:08]** We've been working with Yum for a couple of years now,

**[00:01:08 – 00:01:10]** and we're going to cover a couple of things today.

**[00:01:10 – 00:01:13]** We're going to hear more about their journey from CAM, and then

**[00:01:13 – 00:01:16]** we're going to dive into both the technology decisions as well as the

**[00:01:16 – 00:01:19]** technology capabilities with Joey.

**[00:01:19 – 00:01:21]** Our goal here is to have a great discussion so that you

**[00:01:21 – 00:01:25]** all walk away with some key lessons and some key learnings about

**[00:01:25 – 00:01:29]** what's working, what's challenging, how do I get started in this space,

**[00:01:29 – 00:01:33]** because it's an exciting space and it's incredibly fast-moving.

**[00:01:33 – 00:01:34]** And so with that context,

**[00:01:34 – 00:01:37]** I'm incredibly proud and excited to welcome you,

**[00:01:37 – 00:01:39]** Cam, and hand it over to you.

**[00:01:39 – 00:01:42]** I'm going to stand up because I can't present sitting down.

**[00:01:42 – 00:01:44]** I'm too antsy, and you all know my entire resume.

**[00:01:44 – 00:01:46]** If you want to know my mother's maiden name, I think that's

**[00:01:46 – 00:01:47]** the only thing they left out.

**[00:01:47 – 00:01:49]** It's Cameron, by the way, so you got it.

**[00:01:49 – 00:01:50]** Now you can socially engineer anything You

**[00:01:50 – 00:01:52]** can do anything you want.

**[00:01:52 – 00:01:53]** How many of you have heard of Yum!

**[00:01:53 – 00:01:55]** Brands before walking in this room?

**[00:01:55 – 00:01:57]** All right, a few of you.

**[00:01:57 – 00:02:00]** How many of you have eaten at a Taco Bell, a KFC,

**[00:02:00 – 00:02:01]** or a Habit Burger?

**[00:02:01 – 00:02:02]** Okay, good.

**[00:02:02 – 00:02:04]** If I say Habit Burger, unless you're from California, you

**[00:02:04 – 00:02:05]** don't know what that is.

**[00:02:05 – 00:02:08]** A lot of people get there. But the best way to do this,

**[00:02:09 – 00:02:12]** I think, is just roll a video, tell you a little bit, let you

**[00:02:12 – 00:02:15]** see a little bit about the company.

**[00:02:18 – 00:02:22]** Thank you for watching.

**[00:02:30 – 00:02:38]** I'm the case, I'm on the rhythm, you know we don't stop.

**[00:03:04 – 00:03:07]** Put them in the rhythm, you know we don't stop.

**[00:03:10 – 00:03:12]** My DJ, yo, get on the case.

**[00:03:12 – 00:03:13]** Hypop with the lights on.

**[00:03:13 – 00:03:14]** No fuss, no fight time.

**[00:03:14 – 00:03:17]** Get Hypop. When I'm on the rhythm with a little flow.

**[00:03:17 – 00:03:21]** When I'm on the rhythm, you know, we don't stop, stop, stop, stop.

**[00:03:31 – 00:03:35]** I'm gonna read them, you know, we don't stop.

**[00:03:45 – 00:03:49]** I'm gonna read them, you know, we don't stop.

**[00:03:53 – 00:03:56]** So if you didn't know anything about us before, or maybe you knew

**[00:03:56 – 00:04:00]** about us, but you didn't understand the scale of who we are, just to

**[00:04:01 – 00:04:03]** give you a little sense, we're not gonna drain this slide, but you saw

**[00:04:03 – 00:04:05]** some of these stats in the video.

**[00:04:05 – 00:04:08]** We like to talk about ourselves as the world's largest restaurant

**[00:04:08 – 00:04:10]** company, and that's because we've got over 63,000 units doing

**[00:04:10 – 00:04:14]** over $70 billion in system sales across 155 different countries.

**[00:04:17 – 00:04:21]** The way that breaks down is almost a million franchisee

**[00:04:21 – 00:04:26]** employees, almost 100 million transactions a day, but that's

**[00:04:26 – 00:04:32]** all spread across 1,500 franchisees with about 40 billion in just

**[00:04:32 – 00:04:33]** digital sales alone.

**[00:04:33 – 00:04:36]** And you're like, all right, that's cool, but why is it important?

**[00:04:36 – 00:04:38]** Why is it important to our story here?

**[00:04:38 – 00:04:41]** And it's important to our story here because

**[00:04:41 – 00:04:43]** There's a lot of you in the audience that may be into

**[00:04:43 – 00:04:47]** AI or maybe you're a potential vendor who wants to come talk to

**[00:04:47 – 00:04:49]** me after this little presentation.

**[00:04:49 – 00:04:53]** In fact, I think I checked this morning I had 6,000 emails

**[00:04:53 – 00:04:57]** in my kind of public box and about almost half of them

**[00:04:57 – 00:05:01]** were for vendors with AI in the title that wanted to talk to

**[00:05:01 – 00:05:03]** me about something to do with AI.

**[00:05:03 – 00:05:05]** And when they come to me one of the first questions or

**[00:05:05 – 00:05:09]** set of questions I'll ask is show me where you've got it at scale.

**[00:05:09 – 00:05:12]** Where are you in the AI stack?

**[00:05:12 – 00:05:14]** What do you do today? What differentiates you versus

**[00:05:14 – 00:05:17]** what I can just do myself?

**[00:05:17 – 00:05:21]** And if I deploy you across my system in the next year, what's

**[00:05:21 – 00:05:24]** the first thing that's gonna break?

**[00:05:24 – 00:05:28]** Because doing AI in a lab, it's easy.

**[00:05:28 – 00:05:30]** We've all done it before.

**[00:05:30 – 00:05:33]** In fact, the models aren't the problem anymore, I would argue.

**[00:05:33 – 00:05:37]** In fact, sitting in this front row here is my head of AI, Rocky.

**[00:05:37 – 00:05:40]** And I think, Rock, it took us, what, a month to spin

**[00:05:40 – 00:05:45]** up our first ordering agent for Pizza Hut, give or take.

**[00:05:45 – 00:05:47]** Getting it to work in the lab, getting it to phone call,

**[00:05:48 – 00:05:50]** that wasn't the hard part.

**[00:05:50 – 00:05:53]** Getting it to scale,

**[00:05:54 – 00:06:00]** In a messy, imprecise world, that's hard and that's difficult,

**[00:06:00 – 00:06:01]** and you all see it.

**[00:06:01 – 00:06:04]** In fact, we all have this lab versus real world problem.

**[00:06:04 – 00:06:07]** And we all follow the typical ideas of like, all right,

**[00:06:07 – 00:06:10]** make sure you've got business ownership, make sure you've

**[00:06:10 – 00:06:13]** got executive sponsorship, make sure you have an ROI.

**[00:06:13 – 00:06:15]** Like if I'm telling you anything new about that, you should

**[00:06:15 – 00:06:18]** probably think about another job at this point.

**[00:06:18 – 00:06:20]** We all know those things.

**[00:06:21 – 00:06:24]** But when it comes to YUM and why we have to think about these things

**[00:06:24 – 00:06:28]** differently, it's that first number to start with, because when you've

**[00:06:28 – 00:06:31]** got 100 million transactions a day,

**[00:06:31 – 00:06:35]** And you want to scale an AI, then that scale and speed is exciting,

**[00:06:35 – 00:06:41]** but it also means the failure at machine speed is incredibly risky.

**[00:06:42 – 00:06:44]** It can kill you very, very quickly.

**[00:06:44 – 00:06:47]** The other thing you realize is that when you've got 63,000

**[00:06:47 – 00:06:53]** stores across the globe spread across 1,500 franchisees,

**[00:06:53 – 00:06:56]** would it surprise you to learn that we've got over 500 different

**[00:06:56 – 00:06:59]** point of sale systems?

**[00:06:59 – 00:07:04]** Multiple back of house systems, multiple menu systems, multiple

**[00:07:04 – 00:07:07]** supply chain systems.

**[00:07:07 – 00:07:10]** The tech sprawl is immense.

**[00:07:10 – 00:07:14]** So trying to take something in the lab and then integrate

**[00:07:14 – 00:07:18]** it is a challenge you always have to overcome.

**[00:07:18 – 00:07:21]** The other thing to realize is that if you want to be

**[00:07:21 – 00:07:25]** successful at running a restaurant company in 155 countries for

**[00:07:25 – 00:07:29]** 30 plus years, you're going to be really good at a few things.

**[00:07:29 – 00:07:31]** Obviously, you're going to be good at franchising.

**[00:07:31 – 00:07:34]** Obviously, you're going to be good at branding and

**[00:07:34 – 00:07:37]** marketing, but you also have to be really good at menus.

**[00:07:37 – 00:07:42]** And what I mean by that is we adapt those menus to local tastes.

**[00:07:43 – 00:07:44]** That's how you roll and grow.

**[00:07:44 – 00:07:49]** Now imagine a bot or an AI that has to understand not just different

**[00:07:49 – 00:07:52]** brands, not just different menus, but for the same brand across

**[00:07:52 – 00:07:55]** a set of regions, a completely different menu with a completely

**[00:07:55 – 00:07:57]** different set of dynamics to it.

**[00:07:58 – 00:08:01]** As you cross those regions, which means accuracy becomes

**[00:08:01 – 00:08:03]** really hard, but it's really important because for us, it's not

**[00:08:03 – 00:08:05]** just about getting the order wrong.

**[00:08:05 – 00:08:09]** It's about the brand reputation and the brand risk that goes with that.

**[00:08:09 – 00:08:14]** Because as one of the world's largest franchisors, without

**[00:08:14 – 00:08:18]** our brand, we don't have anything.

**[00:08:19 – 00:08:22]** So that's that one TikTok video away, that one Instagram

**[00:08:22 – 00:08:27]** video away, that one X video away from everything falling apart.

**[00:08:27 – 00:08:31]** And so it creates a unique situation for us that we have to be

**[00:08:31 – 00:08:33]** very cognizant and thoughtful of.

**[00:08:33 – 00:08:37]** So the typical approach that you see.

**[00:08:37 – 00:08:39]** We can't take, which means we have to think about a little

**[00:08:39 – 00:08:42]** bit of a different paradigm here.

**[00:08:42 – 00:08:46]** And it's a subtle distinction, but it's an important distinction.

**[00:08:46 – 00:08:49]** Because the typical way to go do this is I'm going to

**[00:08:49 – 00:08:52]** go spin up a POC, and I'm going to create a thing.

**[00:08:52 – 00:08:55]** And that thing is going to do something, and it's

**[00:08:55 – 00:09:00]** going to be generally monolithic in what it does.

**[00:09:00 – 00:09:03]** But the minute I try to take it out of that situation and

**[00:09:03 – 00:09:06]** have it do something else,

**[00:09:06 – 00:09:09]** It falls down, it just falls down.

**[00:09:10 – 00:09:13]** So the good rules are always the rules, but we don't think

**[00:09:13 – 00:09:16]** in terms of use cases anymore.

**[00:09:16 – 00:09:21]** We think in terms of scalable skills and specific beyond that.

**[00:09:21 – 00:09:25]** And what I mean by that is, let's take an example.

**[00:09:25 – 00:09:26]** If I want to build something, I think

**[00:09:26 – 00:09:29]** of it more like a sports team.

**[00:09:29 – 00:09:32]** What I want is a set of transferable, highly talented

**[00:09:32 – 00:09:35]** agent skills that can do a number of things.

**[00:09:35 – 00:09:39]** To get more specific, I like to use this with my team.

**[00:09:39 – 00:09:43]** I could ask you, build me a baseball pitching robot.

**[00:09:43 – 00:09:46]** And you could probably build me a baseball pitching robot.

**[00:09:46 – 00:09:47]** And that's fine.

**[00:09:47 – 00:09:51]** It'd be great at pitching baseballs over home plate to a catcher's

**[00:09:51 – 00:09:54]** mitt at various speeds, maybe throw some curves, maybe throw some

**[00:09:54 – 00:09:57]** fastballs, maybe some change-ups, that that's what it would do.

**[00:09:57 – 00:10:00]** That's very different from me saying to my team, I need

**[00:10:00 – 00:10:03]** a throwing machine.

**[00:10:04 – 00:10:06]** This machine doesn't care what's in its hand, and it

**[00:10:06 – 00:10:08]** doesn't care what the target is.

**[00:10:08 – 00:10:11]** It's just really good at throwing things.

**[00:10:11 – 00:10:12]** And then I can have a hitting machine, and I can have a

**[00:10:13 – 00:10:14]** catching machine.

**[00:10:14 – 00:10:19]** And so if we do that, instead of thinking in terms of these

**[00:10:19 – 00:10:22]** specific use cases, we think of the terms of capabilities.

**[00:10:22 – 00:10:26]** We have Customer-Facing Agents.

**[00:10:26 – 00:10:29]** That can show up in any number of use cases.

**[00:10:29 – 00:10:31]** We have Team Productivity Agents.

**[00:10:31 – 00:10:34]** That can show up in any number of use cases.

**[00:10:34 – 00:10:39]** And if you do that, what you find, if you've got the right platforms,

**[00:10:39 – 00:10:43]** you can bring these agents to bear at any number of use cases, and

**[00:10:43 – 00:10:45]** that's how you start to get scale.

**[00:10:45 – 00:10:49]** Because you have to think about it just a little bit differently.

**[00:10:50 – 00:10:53]** Now, I've already talked a little bit about

**[00:10:53 – 00:10:55]** Integrations and there's a general rule of thumb when

**[00:10:55 – 00:10:59]** it comes to agents and that is they're only as powerful as

**[00:10:59 – 00:11:03]** the system you let them act within.

**[00:11:03 – 00:11:07]** And that's a general truism that's going to be completely true.

**[00:11:07 – 00:11:11]** And so when you're YUM, and you've got all of these systems

**[00:11:11 – 00:11:15]** across the world, this is why Byte by YUM becomes so

**[00:11:16 – 00:11:20]** incredibly important to us, because this is our proprietary e-comm

**[00:11:21 – 00:11:25]** platform, point-of-sale platform, KDS platform, menu platforms, et

**[00:11:25 – 00:11:29]** cetera, that it all comes together to create this kind of AI flywheel.

**[00:11:29 – 00:11:32]** Because you can look at YUM and you can say, They look.

**[00:11:32 – 00:11:36]** Scale is important, but I learned a really hard lesson, if I can share

**[00:11:36 – 00:11:37]** a story with you real quickly.

**[00:11:37 – 00:11:40]** When I was 16 years old, I grew up playing football,

**[00:11:40 – 00:11:44]** thought I was pretty good at football, made it onto the team.

**[00:11:44 – 00:11:50]** Was sitting in the varsity game and usually played linebacker,

**[00:11:50 – 00:11:51]** but the right tackle got hurt.

**[00:11:51 – 00:11:55]** So the coach is like, look, Davies, I need you to go both ways tonight.

**[00:11:55 – 00:11:56]** I need you to play right tackle.

**[00:11:56 – 00:11:57]** No problem. I can play right tackle.

**[00:11:58 – 00:11:59]** It's easy. How smart was the lineman, right?

**[00:12:00 – 00:12:02]** So I'm going to do it and I'm going to go up there and

**[00:12:02 – 00:12:03]** I'm going to play right tackle.

**[00:12:03 – 00:12:07]** And so I come up to the line for the first play and I look across

**[00:12:07 – 00:12:09]** the line at this defensive tackle.

**[00:12:09 – 00:12:13]** And this is a big human being.

**[00:12:13 – 00:12:18]** He's big. Like, no problem, I've dealt with big before.

**[00:12:18 – 00:12:19]** There are tactics for dealing with big.

**[00:12:19 – 00:12:21]** I'm not gonna stop him, I've just gotta deflect him, I

**[00:12:21 – 00:12:24]** gotta move him, I gotta delay him, it'll be fine.

**[00:12:24 – 00:12:28]** And then the ball snapped, and I found out something else really

**[00:12:28 – 00:12:30]** important about this human being.

**[00:12:30 – 00:12:32]** He was fast.

**[00:12:32 – 00:12:34]** He was really fast.

**[00:12:34 – 00:12:38]** And I spent an hour, four quarters, the longest of my

**[00:12:38 – 00:12:41]** life, getting the crap kicked out of me, getting outrun,

**[00:12:41 – 00:12:43]** getting outgunned by this guy that I just couldn't keep up with.

**[00:12:43 – 00:12:47]** Not because he was so big, because he was so big and fast.

**[00:12:47 – 00:12:49]** And you learn a valuable lesson that's true in sports, it's

**[00:12:49 – 00:12:51]** also true in life.

**[00:12:52 – 00:12:55]** Scale is good, but speed is better.

**[00:12:55 – 00:12:58]** And when you put scale and speed together, that creates

**[00:12:58 – 00:13:01]** a competitive advantage that nobody

**[00:13:01 – 00:13:03]** Can compete with.

**[00:13:03 – 00:13:07]** So we like to think about Byte and our AI as the AI flywheel because

**[00:13:07 – 00:13:13]** we believe the more surfaces we own, the more outcome data we get.

**[00:13:14 – 00:13:17]** The more outcome data we get, the better the experience

**[00:13:18 – 00:13:20]** on our better AI.

**[00:13:20 – 00:13:23]** The better our AI, the better experiences on those surfaces.

**[00:13:23 – 00:13:26]** The better the experiences on those surfaces, the more

**[00:13:26 – 00:13:28]** people want to use those surfaces.

**[00:13:28 – 00:13:31]** The more people that use those surfaces, guess what?

**[00:13:31 – 00:13:33]** The more outcome data I get.

**[00:13:33 – 00:13:35]** And it goes on and on and on.

**[00:13:35 – 00:13:38]** And therefore, our scale and speed of learning and

**[00:13:38 – 00:13:41]** execution can outpace anybody else in the industry.

**[00:13:42 – 00:13:44]** And that's what we're shooting for.

**[00:13:44 – 00:13:47]** So, just to wrap this up, we'll hand it back over to

**[00:13:47 – 00:13:51]** Andrew and Joey, but a few things, key lessons as you

**[00:13:51 – 00:13:54]** think about this from a takeaway.

**[00:13:54 – 00:13:57]** Number one, if you're gonna do AI, people often ask me

**[00:13:57 – 00:14:01]** this in conferences, like, Cam, what's the first thing you do?

**[00:14:01 – 00:14:02]** What's the first thing you do?

**[00:14:03 – 00:14:06]** The first thing you do is governance because there is

**[00:14:06 – 00:14:09]** no intelligence without governance.

**[00:14:09 – 00:14:11]** And I've got a really good friend of mine, Manoj Saxena.

**[00:14:11 – 00:14:13]** I've known Manoj a lot of years.

**[00:14:13 – 00:14:16]** He's the chair of the Responsible AI Institute.

**[00:14:16 – 00:14:19]** His latest startup is a little startup called TrustWise.

**[00:14:19 – 00:14:20]** And I had him in a couple of weeks ago to speak at my

**[00:14:20 – 00:14:23]** AI summit for YUM.

**[00:14:23 – 00:14:26]** And he used this example, because we're all talking about agentic

**[00:14:26 – 00:14:31]** workflows and the exciting and how exciting agentic workflows are.

**[00:14:31 – 00:14:34]** And Minot said, look, you guys are all excited about

**[00:14:34 – 00:14:37]** these agents and what they're going to do, but you don't

**[00:14:37 – 00:14:39]** have the right governance in place.

**[00:14:39 – 00:14:41]** So what you're doing is you're all building these little Chucky's.

**[00:14:41 – 00:14:45]** Remember the movie with Chucky, right?

**[00:14:45 – 00:14:47]** Child's Play. And in one hand's a knife and the other

**[00:14:47 – 00:14:49]** hand's a credit card.

**[00:14:49 – 00:14:51]** And you're sending him loose into the system and he's stabbing

**[00:14:51 – 00:14:53]** and swiping and you don't know what he's doing because

**[00:14:53 – 00:14:55]** you have no agent registries, you have no control.

**[00:14:56 – 00:15:01]** Look, without governance, failure at machine speed is catastrophic.

**[00:15:01 – 00:15:04]** If you're gonna do one thing, start with the governance.

**[00:15:04 – 00:15:10]** The second thing I would say is platforms, not pilots.

**[00:15:10 – 00:15:14]** If you don't have your own AI platform or a good partnership with

**[00:15:14 – 00:15:17]** somebody to go do that, and we can talk about that in a moment, then

**[00:15:17 – 00:15:20]** you need to start there because you start with the AI governance

**[00:15:20 – 00:15:24]** and you start with the platform and that gives you a solid foundation.

**[00:15:24 – 00:15:26]** You can go do a bunch of POCs, you can bring in all the money from

**[00:15:26 – 00:15:30]** the hyperscalers, you can do all of that, but none of it will scale.

**[00:15:30 – 00:15:34]** And none of it will scale safely and cost-effectively

**[00:15:34 – 00:15:36]** if you're not smart.

**[00:15:36 – 00:15:39]** And then thirdly is integration matters.

**[00:15:39 – 00:15:43]** So how you do this and where you do this, picking either

**[00:15:43 – 00:15:45]** to own the platforms,

**[00:15:45 – 00:15:49]** And the system of record and the system of action, or having a small

**[00:15:49 – 00:15:52]** set of partners that you do that with is super important, because

**[00:15:52 – 00:15:56]** otherwise you will die a death by a thousand integration cuts.

**[00:15:56 – 00:16:00]** I've seen it happen again and again and again.

**[00:16:01 – 00:16:03]** Measurement and business impact.

**[00:16:03 – 00:16:05]** Look, there's a, how many of you have seen the movie

**[00:16:05 – 00:16:06]** or read the book, Fight Club?

**[00:16:06 – 00:16:11]** All right, what's the first rule of Fight Club?

**[00:16:11 – 00:16:13]** Don't talk about Fight Club, right?

**[00:16:13 – 00:16:16]** Anybody do CrossFit?

**[00:16:16 – 00:16:17]** No CrossFitters in the room?

**[00:16:17 – 00:16:19]** Anybody know the first rule of CrossFit?

**[00:16:19 – 00:16:22]** You can only talk about CrossFit.

**[00:16:22 – 00:16:25]** And if you have some of you, you know that's true.

**[00:16:25 – 00:16:28]** I'd like to say there's the first rule of measurement

**[00:16:28 – 00:16:32]** is always measure and always talk about measurement.

**[00:16:32 – 00:16:33]** Never stop doing it.

**[00:16:33 – 00:16:38]** Build it into your systems from day one.

**[00:16:38 – 00:16:39]** Testing controls, great.

**[00:16:39 – 00:16:42]** Doing like A-B tests are great.

**[00:16:42 – 00:16:46]** Build a measurement directly into the systems.

**[00:16:46 – 00:16:48]** Continually measure it and continually talk

**[00:16:48 – 00:16:49]** about the measurement.

**[00:16:49 – 00:16:53]** You cannot communicate that enough.

**[00:16:53 – 00:16:57]** I would defy you if anybody finally says, hey, shut up,

**[00:16:57 – 00:16:58]** quit talking about the measurement.

**[00:16:58 – 00:17:00]** I've never had that happen to me.

**[00:17:00 – 00:17:01]** I've had a lot of people ask me about the measurement,

**[00:17:01 – 00:17:03]** ask me how it's going, why don't we have more of it?

**[00:17:03 – 00:17:06]** Where's the ROI? Where are the KPIs?

**[00:17:06 – 00:17:07]** But I've never had somebody tell me they've heard too

**[00:17:07 – 00:17:10]** much about the measurement or they have too much faith in

**[00:17:10 – 00:17:11]** the system and how it's performing.

**[00:17:12 – 00:17:14]** Never happened.

**[00:17:14 – 00:17:17]** And then finally, strong business partners.

**[00:17:17 – 00:17:20]** Look, none of us can do this alone.

**[00:17:21 – 00:17:22]** None of us can do this alone.

**[00:17:22 – 00:17:26]** This business, this space is growing too fast.

**[00:17:26 – 00:17:29]** It's too complex.

**[00:17:29 – 00:17:31]** So this is why, for example, why I'm here today, like

**[00:17:31 – 00:17:34]** why am I standing on stage with NVIDIA today?

**[00:17:34 – 00:17:37]** It's because our partnership with them is core to doing

**[00:17:37 – 00:17:41]** what we need to do, like using domain-specific tools, and

**[00:17:41 – 00:17:44]** there'll be other sessions we'll have, Rocky will be presenting

**[00:17:44 – 00:17:47]** tomorrow on, to talk about some of these tools with Nemotron and how

**[00:17:47 – 00:17:50]** we use NIM for scalable things and how we have access to create these

**[00:17:50 – 00:17:54]** platforms and these capabilities from voice to inference to

**[00:17:54 – 00:17:56]** how we go create synthetic data.

**[00:17:56 – 00:18:00]** But all of having access to all of that speeds us up and

**[00:18:00 – 00:18:03]** helps us be smarter than we would just be by ourselves.

**[00:18:03 – 00:18:05]** And so those are some key takeaways.

**[00:18:05 – 00:18:08]** So with that, I'm gonna turn it over to Andrew to talk a little

**[00:18:08 – 00:18:11]** bit more about open model usage.

**[00:18:11 – 00:18:13]** Thanks, Cam. Before we jump into that.

**[00:18:13 – 00:18:16]** First of all, just thanks for sharing your journey and the story.

**[00:18:16 – 00:18:20]** I wanna click into obviously the technical decisions as well as

**[00:18:20 – 00:18:24]** the capabilities, but before I do that, I think what you've described

**[00:18:24 – 00:18:27]** here is something that probably everyone in this room feels, right?

**[00:18:27 – 00:18:31]** You've probably launched some amount of AI in a lab.

**[00:18:31 – 00:18:35]** You need to scale it into production, but there's complexity

**[00:18:35 – 00:18:37]** around how that's going to survive, how that's going

**[00:18:37 – 00:18:41]** to operate within the risk tolerance of your organization.

**[00:18:41 – 00:18:44]** In retail, consumer goods, QSR, you're probably asking

**[00:18:44 – 00:18:47]** or being asked tough questions about your investment.

**[00:18:47 – 00:18:51]** You have a set amount of precious resources if you have developers

**[00:18:51 – 00:18:56]** or a finite budget you need to spend to enter or grow

**[00:18:56 – 00:18:58]** your presence in this space.

**[00:18:58 – 00:19:01]** And then you've got to do all of this in addition to

**[00:19:01 – 00:19:04]** your core business, just operating and growing your core business

**[00:19:04 – 00:19:05]** for your shareholders.

**[00:19:05 – 00:19:09]** And so I really, I think the question really then becomes, what

**[00:19:09 – 00:19:15]** are some of these core trends that are game changing and essential for

**[00:19:15 – 00:19:18]** Not just technology, but business leaders to appreciate.

**[00:19:18 – 00:19:22]** And this is where we've heard a lot from Jensen in the keynote

**[00:19:22 – 00:19:23]** about open source, right?

**[00:19:23 – 00:19:28]** Open source is much more than just a technical evaluation.

**[00:19:28 – 00:19:33]** It is a complete model paradigm shift in how business leaders

**[00:19:33 – 00:19:37]** need to think about their business because it's really about

**[00:19:37 – 00:19:40]** how do you own your intelligence?

**[00:19:40 – 00:19:44]** How do you control your short-term and your long-term costs?

**[00:19:44 – 00:19:47]** And how do you think through some of the more strategic elements?

**[00:19:47 – 00:19:51]** Joey, you work at the intersection of this space.

**[00:19:51 – 00:19:57]** You're helping cross industries build software from NVIDIA

**[00:19:57 – 00:20:01]** that allows them to transform and accelerate in AI.

**[00:20:01 – 00:20:04]** Can you talk to us a bit about what trends you're seeing in open

**[00:20:04 – 00:20:09]** source, how to think about what impacts that can have for industry?

**[00:20:09 – 00:20:10]** Yeah, thanks.

**[00:20:10 – 00:20:12]** I think, so what's exciting about Yum!

**[00:20:12 – 00:20:16]** Brands is just the scale and the difficulty of the problem.

**[00:20:16 – 00:20:18]** And I think many people can relate.

**[00:20:18 – 00:20:21]** The trends we're seeing in open source, you can see some

**[00:20:21 – 00:20:24]** here on the charts, we see that open source models are becoming

**[00:20:24 – 00:20:25]** better and better and better.

**[00:20:25 – 00:20:27]** And I think we've been really excited to see that.

**[00:20:27 – 00:20:30]** And like these metrics reflect that, the capabilities of

**[00:20:30 – 00:20:31]** what they can do are growing.

**[00:20:31 – 00:20:33]** And we're very excited by that.

**[00:20:33 – 00:20:36]** I think specifically underneath there, we see things like

**[00:20:36 – 00:20:38]** reasoning coming to these models.

**[00:20:38 – 00:20:41]** We see abilities to handle speech and complex scenarios

**[00:20:41 – 00:20:44]** with background noise and different accents and dialogues.

**[00:20:44 – 00:20:47]** I think another thing we're seeing in this space, too, is that as

**[00:20:47 – 00:20:50]** these models gain the capabilities, then companies are able to

**[00:20:50 – 00:20:52]** think about how to scale them up.

**[00:20:52 – 00:20:55]** And so you can think about how to take models and make them smaller,

**[00:20:55 – 00:20:58]** put them on different footprints and different environments.

**[00:20:58 – 00:21:01]** And then also be able to embed your intellectual property,

**[00:21:01 – 00:21:04]** your knowledge, your company's capabilities into these models.

**[00:21:04 – 00:21:07]** And they now essentially start to join your workforce.

**[00:21:07 – 00:21:10]** So I think in kind of the high level here for these open models,

**[00:21:10 – 00:21:13]** the capabilities are increasing significantly year over year.

**[00:21:13 – 00:21:17]** And their ability to then enterprises provide the skill

**[00:21:17 – 00:21:20]** sets and knowledge into those models is also increasing,

**[00:21:20 – 00:21:22]** allowing businesses to have the confidence now to build

**[00:21:22 – 00:21:25]** upon them and bring them into their business and make them theirs.

**[00:21:25 – 00:21:29]** Yeah, and I think your point on scale is so relevant for QSR,

**[00:21:29 – 00:21:30]** for retail, for consumer goods.

**[00:21:30 – 00:21:34]** These are traditionally kind of low-margin or thin-margin

**[00:21:34 – 00:21:39]** industries, and yet the volume and the scale is enormous, right?

**[00:21:39 – 00:21:42]** I mean, in terms of your customer reach, in terms of the number of

**[00:21:42 – 00:21:45]** people eating at your restaurants or shopping at your store,

**[00:21:45 – 00:21:48]** in terms of the sheer volume of employees, right?

**[00:21:48 – 00:21:51]** Team members, store associates, if you look at retail

**[00:21:51 – 00:21:52]** in the U.S., right?

**[00:21:52 – 00:21:57]** You know, I think Walmart's Associates is the largest

**[00:21:57 – 00:21:59]** private employer in the US, right?

**[00:21:59 – 00:22:01]** Second only to the US military, right?

**[00:22:01 – 00:22:04]** So this scale matters a lot in terms of when you think

**[00:22:04 – 00:22:07]** about how do you bring intelligence and maximize the reach and

**[00:22:07 – 00:22:10]** how do you do that efficiently?

**[00:22:10 – 00:22:13]** Let's move on and talk a bit about some of the technology

**[00:22:13 – 00:22:17]** approaches that came to bear in the YUM relationship.

**[00:22:17 – 00:22:20]** And, you know, one of the things, Cam, that I appreciate

**[00:22:20 – 00:22:25]** kind of working across industries and especially with YUM is you

**[00:22:25 – 00:22:28]** guys have a very unique business model as a franchise, you know,

**[00:22:28 – 00:22:30]** predominantly franchise business.

**[00:22:30 – 00:22:33]** And whether you've been in a franchise business, in restaurants,

**[00:22:33 – 00:22:37]** or even in retail, or, you know, how many of you have

**[00:22:37 – 00:22:40]** ever worked in the restaurant business or food service business

**[00:22:40 – 00:22:42]** at any point in your life, right?

**[00:22:42 – 00:22:45]** So if you've been in this business, you appreciate how

**[00:22:45 – 00:22:49]** demanding of an operating environment restaurants can be.

**[00:22:49 – 00:22:52]** It's, you know, I think it's like the retail equivalent

**[00:22:52 – 00:22:56]** of having Black Friday for two hours in the afternoon and three

**[00:22:56 – 00:22:59]** hours at night, 365 days a year.

**[00:23:00 – 00:23:03]** And when you have that type of operating environment,

**[00:23:03 – 00:23:08]** you know, you're usually thinking about, do I have enough people

**[00:23:08 – 00:23:09]** to work at the restaurant today?

**[00:23:10 – 00:23:14]** Do I have enough food and inventory to feed my customers?

**[00:23:14 – 00:23:16]** What am I gonna do with the fryer that's been acting up?

**[00:23:16 – 00:23:20]** You know, the frying machine needs to be fixed by lunch tomorrow.

**[00:23:20 – 00:23:23]** And so when you're in that environment, you're not

**[00:23:23 – 00:23:27]** usually thinking, did I upload the voice transcription

**[00:23:27 – 00:23:30]** logs from the drive-through for the last two weeks?

**[00:23:30 – 00:23:33]** Like that's just not a first order problem.

**[00:23:33 – 00:23:37]** And I think where Yum deserves a lot of credit is you all

**[00:23:37 – 00:23:41]** took what could have been a costly data challenge and

**[00:23:41 – 00:23:47]** data gap, and you turned it into a strategic competitive advantage.

**[00:23:47 – 00:23:50]** And how you did it with your systems thinking, Cameron,

**[00:23:50 – 00:23:53]** your point here is, it wasn't jumping to this second row

**[00:23:53 – 00:23:55]** here of talking about models and data and how to measure.

**[00:23:55 – 00:23:57]** Those are important, but it was actually the first order

**[00:23:57 – 00:24:02]** question of how are we gonna evaluate this at scale?

**[00:24:02 – 00:24:07]** That fundamental question helps unify, I think, business

**[00:24:07 – 00:24:11]** technology in a manner where it became very clear what your

**[00:24:11 – 00:24:15]** constraints are and what trade-offs you wanted to make in terms

**[00:24:15 – 00:24:17]** of these subsequent questions.

**[00:24:17 – 00:24:19]** Yeah, just to add to that, Andrew, I mean, there's a

**[00:24:19 – 00:24:22]** really good example of this, which is what you think you

**[00:24:22 – 00:24:25]** know versus what you learn when you get out there at scale, that you

**[00:24:25 – 00:24:28]** don't see in A-B testing, you don't see in some of the original one.

**[00:24:28 – 00:24:31]** I'll give you voice AI, for example.

**[00:24:31 – 00:24:34]** We had a hypothesis about where it would play out around

**[00:24:35 – 00:24:40]** labor, around upsell, around speed, and in some cases it did, but

**[00:24:40 – 00:24:43]** we saw something really interesting that we never expected when

**[00:24:43 – 00:24:47]** we started to get into 200, 300, 400, 500, 700 plus restaurants.

**[00:24:49 – 00:24:52]** And that there was this consistent, like sometimes you upsell,

**[00:24:52 – 00:24:54]** you got uptake, sometimes you didn't, sometimes speed,

**[00:24:54 – 00:24:56]** sometimes you didn't, it depended on the restaurant.

**[00:24:56 – 00:24:58]** Like restaurants are like people, they're all a little

**[00:24:58 – 00:25:02]** individualistic, but the one consistent thing you saw was

**[00:25:02 – 00:25:06]** the employee satisfaction.

**[00:25:06 – 00:25:09]** Because as you took burden off them, because it is hard, if you've

**[00:25:09 – 00:25:13]** never done that, put a headset on and try to like take an order, talk

**[00:25:13 – 00:25:16]** to somebody, fill a drink order, take cash, hand them an order.

**[00:25:16 – 00:25:19]** Like it's stressful, right?

**[00:25:19 – 00:25:21]** And so like taking that load off.

**[00:25:21 – 00:25:25]** Just reducing that cognitive load, you saw their satisfaction go up.

**[00:25:25 – 00:25:28]** And in an average QSR, not in our system, but in average

**[00:25:28 – 00:25:34]** QSR, you see 300 plus percent changeover on an annual basis.

**[00:25:34 – 00:25:38]** Like reducing that turnover by an order of magnitude of

**[00:25:38 – 00:25:41]** two to three X is huge.

**[00:25:41 – 00:25:46]** It's not just huge in your cost for recruiting, but it's huge because

**[00:25:46 – 00:25:48]** having somebody who's done it,

**[00:25:48 – 00:25:51]** Putting more experience in the restaurant to help run

**[00:25:51 – 00:25:53]** it, to help manage it, et cetera, is just immense.

**[00:25:53 – 00:25:55]** But you'd never see that in a POC.

**[00:25:55 – 00:25:58]** You had to get it out there at scale and see it happen

**[00:25:58 – 00:26:01]** again and again and again and to kind of play it out.

**[00:26:01 – 00:26:04]** And as you do that to your point, then you start getting

**[00:26:04 – 00:26:06]** this data and the results say, ooh, maybe I'm going

**[00:26:06 – 00:26:07]** to think about this differently.

**[00:26:07 – 00:26:10]** Maybe I'm going to actually synthesize it differently.

**[00:26:10 – 00:26:12]** Yeah, that's so on point.

**[00:26:12 – 00:26:14]** I think these lessons that we've learned in terms of

**[00:26:14 – 00:26:15]** the model, right?

**[00:26:15 – 00:26:19]** YUM made a very deliberate transition in the last year towards

**[00:26:19 – 00:26:23]** open source, small language models, as well as a lot of post-training.

**[00:26:24 – 00:26:26]** We're gonna talk a little bit more about that.

**[00:26:26 – 00:26:29]** And key to that was just when they thought about evaluating at scale,

**[00:26:29 – 00:26:34]** one was, as Cam mentioned, Byte, and owning their intelligence was

**[00:26:34 – 00:26:35]** paramount and fundamental, right?

**[00:26:35 – 00:26:39]** They wanted to make sure that what they learned

**[00:26:39 – 00:26:43]** would accumulate and compound rather than kind of renting

**[00:26:43 – 00:26:46]** their intelligence which you know if a model changes or if

**[00:26:46 – 00:26:49]** it's closed you don't you know you kind of lose that when the model

**[00:26:49 – 00:26:52]** switches and clearly this is a very fast moving market the other area

**[00:26:52 – 00:26:56]** that's really fundamental was data and a lot of the work that we're

**[00:26:56 – 00:26:59]** going to talk about today is around synthetic data some of these new

**[00:26:59 – 00:27:02]** These are new capabilities whereby

**[00:27:02 – 00:27:06]** You know, the fact that YUM had some amount of golden data,

**[00:27:06 – 00:27:09]** but not necessarily the 98% of franchises and the breadth that it

**[00:27:09 – 00:27:13]** wanted, they were able to address this with the latest techniques

**[00:27:13 – 00:27:17]** in synthetic data generation to help them actually improve

**[00:27:17 – 00:27:18]** and train models against that.

**[00:27:18 – 00:27:21]** So over 20,000 records with Nemo Data Designer.

**[00:27:21 – 00:27:23]** And the last part that we'll click into here

**[00:27:23 – 00:27:25]** is how did they measure it?

**[00:27:25 – 00:27:28]** And success here goes to what Cam was talking about, the

**[00:27:28 – 00:27:29]** throwing machine.

**[00:27:29 – 00:27:32]** What they really did from an architecture perspective

**[00:27:32 – 00:27:36]** that was unique was they built an LLM harness that enabled them

**[00:27:36 – 00:27:40]** as an organization to introduce intelligence across menus,

**[00:27:40 – 00:27:42]** across brands, across use cases.

**[00:27:43 – 00:27:45]** I think with that, Joey, I'd love to kind of hear your

**[00:27:45 – 00:27:48]** point of view here, because I think a year ago, a lot of enterprises

**[00:27:49 – 00:27:54]** were racing, going very fast with large, large language models.

**[00:27:54 – 00:27:58]** And and in the last year, kind of tell us a bit about your

**[00:27:58 – 00:28:03]** recommendation here around SLMs and open source and LoRa adapters.

**[00:28:03 – 00:28:04]** That's a lot of acronyms.

**[00:28:04 – 00:28:06]** So maybe you could just set it up with just explaining

**[00:28:06 – 00:28:08]** what those are for our audience.

**[00:28:08 – 00:28:12]** Yeah, sure. So I think generally most people start with, say,

**[00:28:12 – 00:28:15]** like an API endpoint, like a model hosted elsewhere.

**[00:28:15 – 00:28:17]** It's a great, easy place to start.

**[00:28:17 – 00:28:19]** You can start in the lab, you get the experience, you get familiar

**[00:28:20 – 00:28:23]** with how the technology works, what it's great at and what it's not.

**[00:28:23 – 00:28:25]** I think, though, for many businesses, they then think about,

**[00:28:25 – 00:28:27]** like, how does this apply to me?

**[00:28:27 – 00:28:29]** How do I take this into my environment?

**[00:28:29 – 00:28:32]** And so with open models and specifically with like Nemotron

**[00:28:32 – 00:28:35]** and Riva, we make these models available in different sizes.

**[00:28:35 – 00:28:37]** And so there are very large ones.

**[00:28:37 – 00:28:39]** We mentioned, Jensen mentioned in our keynote about a

**[00:28:39 – 00:28:40]** Nemotron Ultra coming.

**[00:28:40 – 00:28:41]** It's very large.

**[00:28:41 – 00:28:44]** It'll be the smartest, best open model we have.

**[00:28:44 – 00:28:47]** On the other end of the spectrum, we also have very small models that

**[00:28:47 – 00:28:51]** can run in, say, like edge, power, energy-constrained environments.

**[00:28:51 – 00:28:54]** And we have that breadth because different businesses and different

**[00:28:54 – 00:28:56]** scenarios will need to use different models.

**[00:28:56 – 00:28:58]** And so with that breadth, it allows them to pick and

**[00:28:58 – 00:29:02]** choose where they want to deploy and where they want to customize.

**[00:29:02 – 00:29:04]** In terms of the acronyms, so like fine tuning or LoRa

**[00:29:04 – 00:29:07]** adapters, essentially what we're doing is there's a model

**[00:29:07 – 00:29:10]** that we've published, it's an open model, and then enterprises can

**[00:29:10 – 00:29:14]** take it and they take their data sets, they take the original model,

**[00:29:14 – 00:29:17]** and then they teach that model how to operate in their domain.

**[00:29:17 – 00:29:21]** And so LoRa is a compute-efficient way to be able to influence

**[00:29:21 – 00:29:25]** the model's weights, and you can then make it really efficient

**[00:29:25 – 00:29:27]** at inference deployment too.

**[00:29:27 – 00:29:29]** So when you go into production, you have this small model

**[00:29:29 – 00:29:32]** with a small customization, and it becomes really good at a specific

**[00:29:33 – 00:29:35]** task that matters to that business.

**[00:29:35 – 00:29:37]** I think like kind of moving along that spectrum, say from

**[00:29:37 – 00:29:42]** a year ago to now, the technology has improved to such a great extent

**[00:29:42 – 00:29:47]** that synthetic data generation is now more essentially accessible

**[00:29:47 – 00:29:52]** or available to most companies in the sense that if you have some

**[00:29:52 – 00:29:55]** We'll say seed data or key data that is a subject matter

**[00:29:55 – 00:29:57]** expert has labeled something core to your business.

**[00:29:57 – 00:30:00]** With NeMo Data Designer, we can now grow that data set

**[00:30:00 – 00:30:01]** and make it much larger.

**[00:30:01 – 00:30:04]** So when we go to train the model, it has much larger data

**[00:30:04 – 00:30:06]** set to sample from and learn from.

**[00:30:06 – 00:30:09]** And I think 20,000 is a great accomplishment.

**[00:30:09 – 00:30:11]** I think going forward, we would expect to see people

**[00:30:11 – 00:30:14]** be able to do more of that, and you can imagine that applies to voice,

**[00:30:14 – 00:30:17]** it applies to text, it applies to reasoning, it applies across these

**[00:30:18 – 00:30:21]** use cases, and enterprises often have, I think like they mentioned,

**[00:30:21 – 00:30:22]** many of the integrations.

**[00:30:22 – 00:30:24]** You can imagine all these systems, they all have different

**[00:30:24 – 00:30:28]** APIs, different tool calling, like there's so much complexity

**[00:30:28 – 00:30:31]** there, and the people who know that best is YUM.

**[00:30:31 – 00:30:32]** They're the experts at that.

**[00:30:32 – 00:30:34]** And so being able to enable them with these tools where

**[00:30:34 – 00:30:36]** they can take an open model and that model then becomes

**[00:30:37 – 00:30:39]** an expert in how to interact with their systems, how to interact

**[00:30:39 – 00:30:42]** with their partners, how to make the right calls at the right time.

**[00:30:42 – 00:30:44]** And so I think we'll see more of that going forward with

**[00:30:44 – 00:30:48]** some of these new technologies and tools coming out.

**[00:30:48 – 00:30:52]** Let's go one click further and talk about tool calling architecture.

**[00:30:52 – 00:30:57]** This is GTC, so we're going to go deep into the tech.

**[00:30:57 – 00:30:59]** And generally, this is the part of the presentation where,

**[00:30:59 – 00:31:01]** if you're a business leader, you take a screenshot and

**[00:31:01 – 00:31:05]** you're thinking, I'm going to hand this to my CTO or my head of AI.

**[00:31:05 – 00:31:08]** But I actually would implore you to actually walk away from

**[00:31:08 – 00:31:11]** this as this is actually the most important slide for the business.

**[00:31:11 – 00:31:13]** What I mean by that is,

**[00:31:13 – 00:31:19]** What YUM introduced in their evolution was a movement from

**[00:31:19 – 00:31:23]** thinking about is this model giving me the best answer and having

**[00:31:23 – 00:31:30]** 10 different interpretations of yes or no into a scientific, measurable

**[00:31:30 – 00:31:35]** way of actually evaluating and training models for success.

**[00:31:35 – 00:31:36]** And what do we mean by that?

**[00:31:36 – 00:31:40]** Well, what YUM did that was great was they evolved into a set of

**[00:31:40 – 00:31:44]** Decision points and flows, and this was the method they applied.

**[00:31:44 – 00:31:46]** So, you know, we've all heard the word workflow.

**[00:31:46 – 00:31:49]** Well, you know your business best.

**[00:31:49 – 00:31:53]** And which workflows are the most valuable for your business?

**[00:31:53 – 00:31:56]** A flow is really just a chain of decisions, which

**[00:31:56 – 00:31:57]** are tool calls, right?

**[00:31:57 – 00:32:01]** They're independent, they're binary, and by defining

**[00:32:01 – 00:32:05]** these specific decision moments within a workflow,

**[00:32:05 – 00:32:10]** you can then actually evaluate, generate data, and train a

**[00:32:10 – 00:32:15]** model specifically for a successful outcome for your business.

**[00:32:15 – 00:32:17]** So, you know, here on the left, you can see, right,

**[00:32:17 – 00:32:20]** this is a classic, you know, example of just ordering food.

**[00:32:20 – 00:32:23]** And for many of us, maybe that's a very simple order, right?

**[00:32:23 – 00:32:27]** Maybe you're always in Nacho Bell's Grande from Taco Bell with a Pepsi.

**[00:32:27 – 00:32:31]** But for a lot of us, like myself included, I'm definitely

**[00:32:31 – 00:32:34]** going to have a kid that's going to modify that order, right?

**[00:32:34 – 00:32:39]** No, no lettuce in the taco, you know, a lot of too much dessert.

**[00:32:40 – 00:32:42]** And so in that case, when you look at this workflow

**[00:32:42 – 00:32:46]** from finding a product, to applying a modification, to

**[00:32:46 – 00:32:49]** adding it to cart, to submitting an order, each of these became

**[00:32:49 – 00:32:51]** very discrete decision points.

**[00:32:51 – 00:32:55]** And this is where that synthetic data generation and the training

**[00:32:55 – 00:33:00]** of these tool calls became so fundamental for the SLMs.

**[00:33:00 – 00:33:02]** This is where, you know, you could solve the costly

**[00:33:02 – 00:33:03]** data problem, right?

**[00:33:03 – 00:33:08]** Synthetic data suddenly produced a corpus of information that

**[00:33:08 – 00:33:12]** was sizable enough for Yum to develop the intelligence around it.

**[00:33:12 – 00:33:15]** And not only did they do that for voice, but it's really about

**[00:33:15 – 00:33:20]** architecting scalable tool calling architecture across multiple

**[00:33:20 – 00:33:22]** use cases, across multiple brands.

**[00:33:23 – 00:33:24]** It's a really good example.

**[00:33:24 – 00:33:29]** You think, what's simpler than just ordering a taco?

**[00:33:30 – 00:33:31]** It seems pretty simple, right?

**[00:33:31 – 00:33:34]** And Rocky's laughing in the front row because you start

**[00:33:34 – 00:33:39]** to understand along the process, once you get intent, understanding

**[00:33:39 – 00:33:41]** intent's not hard.

**[00:33:41 – 00:33:44]** It's getting the thing to do the right thing and stay

**[00:33:44 – 00:33:45]** within its bounds.

**[00:33:45 – 00:33:50]** And so like ordering a taco, but what if I want to order a bundle?

**[00:33:50 – 00:33:52]** Well, depending on how that menu system's architected, that's

**[00:33:52 – 00:33:55]** actually a different tool call.

**[00:33:55 – 00:33:58]** What if I wanted to recognize that you've essentially ordered

**[00:33:58 – 00:34:03]** a bundle and just make it a bundle for you to save time?

**[00:34:03 – 00:34:05]** That's a new tool call.

**[00:34:05 – 00:34:08]** That's a separate thing. It's a separate skill.

**[00:34:08 – 00:34:10]** And so it's a really good example of what Rock and his

**[00:34:10 – 00:34:13]** team are doing as we think about the architecture and how do I make

**[00:34:13 – 00:34:15]** it scalable against the tool calls?

**[00:34:15 – 00:34:19]** Then I can do multiple things with it because that problem's

**[00:34:19 – 00:34:21]** no different than trying to figure out how to bundle a pizza.

**[00:34:21 – 00:34:24]** That's no different than trying to bundle chicken, which we

**[00:34:24 – 00:34:28]** could also make other applications if we had all day.

**[00:34:28 – 00:34:30]** I won't do it to go into, there are actually similar

**[00:34:30 – 00:34:32]** problems There's a bunch of words that show up in some of my

**[00:34:32 – 00:34:35]** Enterprise workflows, right?

**[00:34:35 – 00:34:36]** When do I make this decision?

**[00:34:36 – 00:34:39]** How do I tell the call? So it's building it in a way that we

**[00:34:39 – 00:34:42]** can do that, create the synthetic data to train it to do it.

**[00:34:42 – 00:34:43]** And then all I've got to do is create the data and train

**[00:34:43 – 00:34:46]** it on that specific thing because it already knows how to throw.

**[00:34:46 – 00:34:48]** It just needs to know what it's holding.

**[00:34:48 – 00:34:51]** Yeah, it feels like you're back in grammar school and

**[00:34:51 – 00:34:55]** having to diagram a sentence a bit, but in many ways that

**[00:34:55 – 00:34:56]** is the right metaphor, right?

**[00:34:56 – 00:34:59]** As a business is trying to figure out what is that

**[00:34:59 – 00:35:01]** specific decision point?

**[00:35:01 – 00:35:03]** What are the parameters that matter?

**[00:35:03 – 00:35:05]** What does success look like?

**[00:35:06 – 00:35:08]** Joey, I wonder if you could talk a little bit about, you

**[00:35:08 – 00:35:11]** know, not just Nemotron, but maybe some of the microservices

**[00:35:11 – 00:35:16]** that help enterprises on this sort of decision point flow approach

**[00:35:16 – 00:35:17]** as they approach this space.

**[00:35:17 – 00:35:21]** Yeah, definitely. I think tool calling is a great example.

**[00:35:21 – 00:35:25]** Any business that has software will have APIs and have tool

**[00:35:25 – 00:35:28]** calling, so it's a pretty generally applicable use case.

**[00:35:28 – 00:35:31]** The way we think about the software stack is we think starting

**[00:35:31 – 00:35:35]** from data, which is like in these scenarios, a crunchy taco and

**[00:35:35 – 00:35:39]** looks like beef and size and price, like you need to have good data.

**[00:35:39 – 00:35:41]** And so at the beginning of this pipeline, we have Nemo

**[00:35:41 – 00:35:44]** Data Designer, which is great for the synthetic data generation.

**[00:35:45 – 00:35:47]** If you're looking at more complex or large scale training, we also

**[00:35:47 – 00:35:49]** have Nemo Data Curator as well.

**[00:35:49 – 00:35:52]** And we have different modalities from text and image and video.

**[00:35:52 – 00:35:53]** But essentially, there's tooling there to get your data in

**[00:35:54 – 00:35:56]** a great place where the model can learn from it.

**[00:35:56 – 00:35:58]** So that's kind of one of the first stages is getting the

**[00:35:58 – 00:36:00]** data ready, getting the right data.

**[00:36:00 – 00:36:02]** If you only have a little bit, growing that data set,

**[00:36:02 – 00:36:04]** if there are gaps, filling that in.

**[00:36:04 – 00:36:06]** The next stage is around customization.

**[00:36:06 – 00:36:10]** And so in NIMO, we have the ability to do both the pre-training

**[00:36:10 – 00:36:11]** and the post-training.

**[00:36:11 – 00:36:13]** In post-training, we have a variety of techniques.

**[00:36:13 – 00:36:15]** So there's LoRa, which is very compute efficient.

**[00:36:15 – 00:36:18]** There's more complex things like doing reinforcement learning

**[00:36:18 – 00:36:20]** that I think many will move towards in the future.

**[00:36:20 – 00:36:23]** But once you're able to go through these training stages, you

**[00:36:23 – 00:36:25]** then have a model or a candidate.

**[00:36:25 – 00:36:27]** And then after that, we do evaluation.

**[00:36:27 – 00:36:28]** And so we have Nemo Evaluator.

**[00:36:28 – 00:36:30]** And essentially, now you measure the improvement.

**[00:36:30 – 00:36:32]** So I have a model at the beginning.

**[00:36:32 – 00:36:35]** I took it through this process of customizing data, training

**[00:36:35 – 00:36:36]** it, and now measuring it.

**[00:36:36 – 00:36:38]** And so at the end, ideally, you're able to see the

**[00:36:38 – 00:36:41]** accuracy improvement in customizing that model.

**[00:36:41 – 00:36:44]** And that's kind of a decision point of, is this good enough?

**[00:36:44 – 00:36:45]** Can I try it and put it out?

**[00:36:45 – 00:36:48]** Or are there scenarios where it's not doing well enough and

**[00:36:48 – 00:36:49]** I need to augment and improve it?

**[00:36:49 – 00:36:51]** And then once you finish with that, the software we have

**[00:36:51 – 00:36:54]** at the end there is NIM, which is for inference.

**[00:36:54 – 00:36:56]** And so once you've finished customizing these models,

**[00:36:56 – 00:36:59]** you then now can put it in a production environment.

**[00:36:59 – 00:37:01]** And these NIMs run across the NVIDIA infrastructure,

**[00:37:02 – 00:37:04]** so you can go small scale at the edge, you can go large

**[00:37:04 – 00:37:07]** scale in the data center, the same software stack runs everywhere.

**[00:37:07 – 00:37:10]** And that applies across Nemo as well.

**[00:37:10 – 00:37:13]** Yeah, I think the main takeaway, thanks Joey, is tool call

**[00:37:13 – 00:37:16]** correctness is the product, right?

**[00:37:16 – 00:37:19]** That's a phrase that Rocky, the head of data science

**[00:37:19 – 00:37:20]** from YAM, coined.

**[00:37:20 – 00:37:23]** And it's ironic, because we talked about diagramming grammar.

**[00:37:23 – 00:37:26]** Tool call correctness feels like a made-up word, but it

**[00:37:26 – 00:37:27]** is the perfect word.

**[00:37:27 – 00:37:30]** Right, in terms of that is the product that you want

**[00:37:30 – 00:37:32]** to develop in your organization.

**[00:37:32 – 00:37:33]** Let's talk real quick about results.

**[00:37:33 – 00:37:35]** Obviously, you know, when you do AI, it's important to

**[00:37:36 – 00:37:38]** show results, demonstrable results.

**[00:37:38 – 00:37:42]** And as you move through in terms of choosing technology that can

**[00:37:42 – 00:37:45]** help you, obviously you want to see that improve both in terms of your

**[00:37:45 – 00:37:47]** performance as well as your cost.

**[00:37:47 – 00:37:50]** And so, you know, with the techniques that

**[00:37:50 – 00:37:51]** were described today,

**[00:37:51 – 00:37:54]** You know, Yum saw effectively a three times improvement

**[00:37:55 – 00:37:58]** off of a base model with tool calling and LoRa fine-tuning,

**[00:37:58 – 00:38:01]** right, getting to 85% accuracy.

**[00:38:02 – 00:38:05]** They saw 100% function match achieved for complex

**[00:38:05 – 00:38:07]** multi-step ordering.

**[00:38:07 – 00:38:10]** And importantly, especially for QoSR and retail, 20

**[00:38:10 – 00:38:14]** times lower inference cost versus general-purpose

**[00:38:14 – 00:38:16]** third-party closed models.

**[00:38:16 – 00:38:18]** This is fundamental, right? This is really, really key

**[00:38:18 – 00:38:21]** to scaling and figuring out how to continue to make improvement

**[00:38:21 – 00:38:22]** in your core business.

**[00:38:23 – 00:38:25]** I think with that, maybe, Cam, you can you can help us wrap here

**[00:38:25 – 00:38:30]** in terms of, you know, where you guys started, what's what's core

**[00:38:30 – 00:38:33]** to you as well as where where it's been deployed and the outcomes.

**[00:38:33 – 00:38:35]** Yeah, we'll do this fast because we're out of time.

**[00:38:35 – 00:38:40]** But so a couple of things on the left, no matter what,

**[00:38:41 – 00:38:42]** do not give up your data.

**[00:38:42 – 00:38:44]** Do not give up your orchestration.

**[00:38:44 – 00:38:46]** I don't care who you partner with.

**[00:38:46 – 00:38:48]** Those are your intellectual property.

**[00:38:48 – 00:38:50]** That's what matters the most.

**[00:38:50 – 00:38:52]** No matter who you partner with, don't give them up.

**[00:38:53 – 00:38:55]** But choose a great partner and then focus on the economics.

**[00:38:56 – 00:38:58]** And on the right side, what you're gonna see is just a quick example

**[00:38:58 – 00:39:02]** when I talk about this, when how these voice agents come to life in

**[00:39:02 – 00:39:04]** different places across the globe.

**[00:39:04 – 00:39:06]** So it can come to life, those skills can come to

**[00:39:06 – 00:39:10]** life in a customer service agent sitting in India.

**[00:39:10 – 00:39:13]** It can come to life in a streamlined restaurant operations

**[00:39:13 – 00:39:16]** agent sitting in the UK, an MEA.

**[00:39:17 – 00:39:19]** And it can come to life in a set of corporate AI agents that

**[00:39:19 – 00:39:21]** help with knowledge management.

**[00:39:21 – 00:39:24]** The same set of skills, same set of components, some different

**[00:39:24 – 00:39:29]** componentized that come to life sitting on our Crave AI platform.

**[00:39:29 – 00:39:31]** In partnership with NVIDIA and the tools that we stack.

**[00:39:31 – 00:39:33]** So that's sort of how it starts to come.

**[00:39:33 – 00:39:36]** And the key thing is, like I mentioned, built in weeks,

**[00:39:37 – 00:39:38]** not quarters, not years.

**[00:39:38 – 00:39:40]** Because if you get the infrastructure right,

**[00:39:40 – 00:39:44]** you get the tools right, you can be big and fast at the same time.

**[00:39:44 – 00:39:46]** Thanks, Cameron. I love how you tied it back to that first

**[00:39:46 – 00:39:47]** principle of speed.

**[00:39:47 – 00:39:51]** It's really important to learn intelligently, be thoughtful about

**[00:39:51 – 00:39:55]** speed, and how thoughtful Yum has been with Byte in their approach.

**[00:39:55 – 00:39:59]** Thank you all for coming today and for joining our session,

**[00:39:59 – 00:40:00]** and we hope you enjoyed it.