# Agentic AI in Production: From Open Models to Agents You Own

## Abstract

Key takeaways Learn about new open models, tools, skills, and secure runtimes in the NVIDIA Agent Toolkit. Understand best practices for building, deploying, and optimizing specialized agents in production. Through real-world examples, explore ways agentic AI can be faster, safer, and more efficient.

## AI Summary

- The presentation covered the advancements in building agentic applications using NVIDIA's open models and tools, emphasizing the importance of model selection and customization.
- NVIDIA introduced NeMo Switchyard, an open library for routing tasks to appropriate models, which helped reduce costs and improve efficiency by using a combination of open and proprietary models.
- The Agent Knowledge Foundation was announced, which included models and libraries to help enterprises connect AI agents to their structured and unstructured data, enabling more context-aware and intelligent agents.
- OpenShell, a security layer for AI agents, was discussed, highlighting its role in controlling and securing agent behavior, ensuring data privacy and preventing unintended actions.
- The presentation showcased examples of how partners like Siemens and Dassault Systems used NVIDIA's tools to optimize their workflows, from design and simulation to production planning, significantly reducing time and effort.
- NVIDIA emphasized the importance of collaboration with partners and the broader ecosystem to improve and customize open models, ensuring they meet specific business needs across various industries.

## Transcript

**[00:00:07 – 00:00:09]** Hi, everyone.

**[00:00:09 – 00:00:13]** Welcome. Welcome to our Agentic AI and Claws track.

**[00:00:14 – 00:00:15]** My name is Reshmi Amarnath.

**[00:00:15 – 00:00:19]** I'm the VP of Enterprise Marketing in EMEA, and I'm going to

**[00:00:19 – 00:00:22]** be your host for this track today.

**[00:00:22 – 00:00:26]** First of all, congratulations on having made it through the crowds.

**[00:00:26 – 00:00:28]** Well done, you're really resilient.

**[00:00:28 – 00:00:29]** Good job.

**[00:00:29 – 00:00:30]** Great to see all of this interest here.

**[00:00:31 – 00:00:34]** I just want to make a few quick announcements before

**[00:00:34 – 00:00:36]** handing over to our first speakers.

**[00:00:36 – 00:00:40]** We have three sessions today, three great sessions.

**[00:00:40 – 00:00:42]** The first one is about to start here.

**[00:00:42 – 00:00:45]** The second one is going to have a room change.

**[00:00:45 – 00:00:49]** We have Shopify and Nebious, also an excellent session,

**[00:00:49 – 00:00:51]** which will be in room five.

**[00:00:51 – 00:00:54]** So we'll have to move from here to the other room, which

**[00:00:54 – 00:00:57]** is behind the Build-A-Clause track.

**[00:00:57 – 00:00:58]** It's easily sign posted.

**[00:00:58 – 00:01:00]** You should be able to find it.

**[00:01:00 – 00:01:03]** And then we come back here for the last session in this track

**[00:01:03 – 00:01:06]** today, which is by Hugging Face.

**[00:01:06 – 00:01:09]** We're gonna be back here again tomorrow for more

**[00:01:09 – 00:01:13]** deep dives, and we look forward to seeing all of you here.

**[00:01:13 – 00:01:16]** And one more announcement I want to make, if you would

**[00:01:16 – 00:01:22]** like to listen to this, get closed captions in German, you

**[00:01:22 – 00:01:26]** can go into the GTC app and choose closed captions, and you have

**[00:01:26 – 00:01:29]** the option of English or German.

**[00:01:29 – 00:01:33]** So, I'd now like to hand over to our first presenters of

**[00:01:33 – 00:01:38]** today, NVIDIA's Carrie Briski, who's our VP of Generative AI

**[00:01:38 – 00:01:43]** Software, and Justin Boitano, who's our VP of Enterprise AI, to talk

**[00:01:43 – 00:01:46]** about Enterprise AI in production.

**[00:01:46 – 00:01:49]** Hello.

**[00:01:49 – 00:01:54]** Good afternoon. Thank you guys all for coming to GTC.

**[00:01:54 – 00:01:59]** So to quickly introduce my friend Kerry, Kerry runs all

**[00:01:59 – 00:02:03]** of our, I'll call it, deep technology around open models

**[00:02:03 – 00:02:05]** and building and training models.

**[00:02:05 – 00:02:07]** And I lead really more of the go-to-market, working

**[00:02:07 – 00:02:09]** with partners at NVIDIA.

**[00:02:09 – 00:02:13]** We've hit this point in the industry where AI, though,

**[00:02:13 – 00:02:15]** has become insanely useful.

**[00:02:16 – 00:02:18]** We saw it starting to happen, I think, in the

**[00:02:18 – 00:02:20]** beginning of the year.

**[00:02:21 – 00:02:24]** We saw this moment where applications, open harnesses

**[00:02:24 – 00:02:28]** like OpenClaw started to be used using Frontier models.

**[00:02:28 – 00:02:32]** But now, both Open models and Frontier model adoption

**[00:02:32 – 00:02:33]** is just exploding.

**[00:02:33 – 00:02:39]** The token growth that we see has increased 25x.

**[00:02:39 – 00:02:45]** And both the usage of closed Frontier models and Open models

**[00:02:45 – 00:02:49]** is really taking hold across every industry in many different

**[00:02:49 – 00:02:55]** applications, in every country and in every industry.

**[00:02:55 – 00:03:01]** The basis or the foundation of these new agentic applications

**[00:03:01 – 00:03:03]** looks a lot like what you see here on the screen.

**[00:03:03 – 00:03:08]** The models are a big part of the conversation, and rightfully so.

**[00:03:08 – 00:03:10]** That is the foundational intelligence for

**[00:03:10 – 00:03:11]** these applications.

**[00:03:11 – 00:03:15]** But a model, if you ask it a question, just gives you an answer.

**[00:03:15 – 00:03:18]** It's not until you connect that model into a harness

**[00:03:18 – 00:03:23]** and you have it on auto run, and it starts to call into your knowledge

**[00:03:23 – 00:03:27]** and your tools, does it start to do meaningful work for your business.

**[00:03:27 – 00:03:29]** And so what we want to do today is spend some time talking

**[00:03:29 – 00:03:33]** about The advancements across this entire stack.

**[00:03:33 – 00:03:36]** Jensen talked about this today, there's sort of five key layers

**[00:03:36 – 00:03:41]** to building agentic applications, and we'll go into great detail

**[00:03:41 – 00:03:47]** in the advancements across models, harnesses, secure runtimes,

**[00:03:47 – 00:03:52]** knowledge and tools that you can build into these applications.

**[00:03:52 – 00:03:57]** But as I mentioned, there's a range of harnesses that you can use.

**[00:03:57 – 00:04:01]** At NVIDIA, we are huge users of Codex.

**[00:04:01 – 00:04:02]** I see my friend Matt in the front row.

**[00:04:03 – 00:04:05]** So yes, we use Codex all the time.

**[00:04:05 – 00:04:07]** We use Clod as well.

**[00:04:07 – 00:04:11]** We also use many of these open harnesses.

**[00:04:11 – 00:04:14]** And we try and use sort of the right models and the right

**[00:04:14 – 00:04:17]** harness for the right job.

**[00:04:17 – 00:04:22]** We have, you know, so many more use cases than we can,

**[00:04:22 – 00:04:25]** you know, just rely on one model family or one harness for.

**[00:04:25 – 00:04:27]** So we want to use all of them.

**[00:04:27 – 00:04:30]** But we'll sort of talk about these, like, all the advancements, all

**[00:04:30 – 00:04:33]** the ways to advance these agents.

**[00:04:33 – 00:04:36]** I guess the first part is really, like, what model do you choose?

**[00:04:36 – 00:04:38]** And so, Kerry, maybe you want to add some color here.

**[00:04:38 – 00:04:40]** Yeah, thank you, Justin.

**[00:04:40 – 00:04:43]** As Justin said, token usage is really growing.

**[00:04:43 – 00:04:46]** I'm really surprised how I have to educate people on

**[00:04:46 – 00:04:49]** it's not an either-or situation, it's not a proprietary closed

**[00:04:49 – 00:04:51]** model versus an open.

**[00:04:51 – 00:04:53]** Just like, this is a new type of software application.

**[00:04:53 – 00:04:56]** I mean, to me, a harness is a software application, and

**[00:04:56 – 00:04:59]** so when you're writing a software application, do you import

**[00:04:59 – 00:04:59]** just one library?

**[00:04:59 – 00:05:01]** The answer is no.

**[00:05:01 – 00:05:03]** You import many libraries to get the job done.

**[00:05:03 – 00:05:05]** And so you're seeing this with models, but then we get the

**[00:05:05 – 00:05:09]** question of how do you choose the right model within the application?

**[00:05:09 – 00:05:13]** And so we came up with an open library called NeMo Switchyard.

**[00:05:13 – 00:05:17]** We're integrating it into the router ecosystem, so we're

**[00:05:17 – 00:05:20]** working with a lot of router partners, and some applications

**[00:05:20 – 00:05:24]** and harnesses have their own routing technology within them.

**[00:05:24 – 00:05:26]** And we have all kinds of routing strategies, whether

**[00:05:26 – 00:05:29]** it's Classification, to be able to tell what kind

**[00:05:29 – 00:05:31]** of task to route to which model.

**[00:05:31 – 00:05:36]** An agent stage, which stage of a long-running, multi-turned,

**[00:05:36 – 00:05:40]** multi-task, agentic task are you doing, and do you need to route to

**[00:05:40 – 00:05:43]** a different model, a different part of the stage, or even escalation?

**[00:05:43 – 00:05:46]** And so when you're doing an agentic task and you reach

**[00:05:46 – 00:05:49]** a point, you're like, oh crap, this model that I've routed to is

**[00:05:49 – 00:05:51]** actually not getting the job done, because you're verifying I need

**[00:05:51 – 00:05:57]** to now go route it and escalate it up to a larger model, more,

**[00:05:57 – 00:06:00]** just a bigger, more proprietary, maybe even a proprietary model.

**[00:06:00 – 00:06:03]** And what's interesting about Switchyard, if you're familiar

**[00:06:03 – 00:06:06]** with Switchyard or maybe not, maybe you're familiar with our

**[00:06:06 – 00:06:08]** inference library called Dynamo.

**[00:06:08 – 00:06:13]** And Dynamo is a way to be able to use the best of your capacity,

**[00:06:13 – 00:06:16]** utilization of your capacity, distributed inference serving.

**[00:06:16 – 00:06:20]** And so I'm going to show you a little demo about how, when

**[00:06:20 – 00:06:23]** maybe you have a coding task, I think this is actually on Talbench.

**[00:06:23 – 00:06:25]** It's actually, it's Terminal Bench.

**[00:06:25 – 00:06:27]** So if you're familiar with Terminal Bench, it's a set

**[00:06:27 – 00:06:31]** of tasks to get things done within a terminal, and then it scores it.

**[00:06:31 – 00:06:34]** And so there's a lot of tasks within this benchmark that

**[00:06:34 – 00:06:37]** you don't really need to route to a really large model.

**[00:06:37 – 00:06:42]** And so you want to be able to do the useful work across many models,

**[00:06:42 – 00:06:45]** and what, once you choose, maybe I'll go back and even replay it.

**[00:06:45 – 00:06:47]** Am I able to do that?

**[00:06:47 – 00:06:50]** Once you're able to choose which model, then you can

**[00:06:50 – 00:06:55]** even take advantage of capacity on even an older generation of GPU.

**[00:06:55 – 00:06:58]** So showing the durability of our GPUs relative to a

**[00:06:58 – 00:07:01]** smaller model, and then you're able to be able to get more

**[00:07:01 – 00:07:03]** tasks done on that capacity.

**[00:07:03 – 00:07:07]** And so you're able to hear the 54% more tasks per hour.

**[00:07:09 – 00:07:13]** So that's kind of exciting. Nemo switchers have really great

**[00:07:13 – 00:07:15]** adoption, really great success.

**[00:07:15 – 00:07:18]** And we would love anyone to, it's open, so you can give

**[00:07:18 – 00:07:22]** us some routing strategies back to our repository.

**[00:07:22 – 00:07:23]** And just here is another one.

**[00:07:23 – 00:07:27]** I think this is, I don't know if this is PinchBench, but

**[00:07:28 – 00:07:32]** basically showing that with a closed model, the cost in

**[00:07:32 – 00:07:35]** order to get this total benchmark done reached a certain accuracy.

**[00:07:35 – 00:07:39]** We were able to use Switchyard to use one open model plus

**[00:07:39 – 00:07:40]** a proprietary model.

**[00:07:40 – 00:07:43]** We got close to the accuracy, but much less cost.

**[00:07:43 – 00:07:45]** But then when we showed, we added even more open models

**[00:07:45 – 00:07:46]** that were task specific.

**[00:07:46 – 00:07:49]** So we were able to use a ton of models to be able to get

**[00:07:49 – 00:07:54]** a higher accuracy going towards that efficiency frontier, higher

**[00:07:54 – 00:07:58]** accuracy, but much less cost.

**[00:07:58 – 00:08:02]** Yeah, so I think that trend we see really across every

**[00:08:02 – 00:08:05]** company, across every industry, is often you want to start

**[00:08:05 – 00:08:09]** with, I'll call it frontier general intelligence, to try and solve

**[00:08:09 – 00:08:12]** whatever business problem is that you're trying to build towards.

**[00:08:12 – 00:08:15]** But basically, as you understand the problem that you're trying

**[00:08:15 – 00:08:17]** to solve, you then want to think through, like, how can I

**[00:08:17 – 00:08:22]** reduce cost, how can I potentially use a family of open models

**[00:08:22 – 00:08:25]** to offset that cost, and so obviously that's going to get you

**[00:08:25 – 00:08:28]** the best intelligence per dollar.

**[00:08:28 – 00:08:31]** And you can build really domain-specific intelligence

**[00:08:31 – 00:08:35]** that outperforms the frontiers on specialized tasks.

**[00:08:35 – 00:08:38]** To facilitate this, we then create families of models.

**[00:08:38 – 00:08:42]** We create language models that we call Nemotron.

**[00:08:42 – 00:08:49]** We have specific models, like RFD4 Proteina for protein

**[00:08:49 – 00:08:51]** and drug discovery.

**[00:08:51 – 00:08:54]** We launched here a new family of models that are called Kumo.

**[00:08:54 – 00:08:58]** These are tabular foundation models, which we'll talk more

**[00:08:58 – 00:08:59]** about in a minute.

**[00:08:59 – 00:09:06]** Cosmos is for physical AI, and then Alpamayo is for self-driving cars.

**[00:09:06 – 00:09:10]** And so NVIDIA believes open models are critical to allow

**[00:09:10 – 00:09:14]** every company, every country, every industry build this

**[00:09:14 – 00:09:19]** domain-specific intelligence and participate in the AI economy.

**[00:09:19 – 00:09:21]** We put not only the model weights out there, but the

**[00:09:21 – 00:09:25]** data sets, and we publish all the techniques so that anybody

**[00:09:25 – 00:09:29]** can participate in building their own domain-specific intelligence

**[00:09:29 – 00:09:33]** to solve their business problems.

**[00:09:33 – 00:09:37]** So, Nemotron is our family of open, efficient, intelligent models,

**[00:09:37 – 00:09:41]** and, like Justin said, we have many different versions of the model

**[00:09:41 – 00:09:45]** of different sizes for different tasks for For the job to be done.

**[00:09:45 – 00:09:49]** It's kind of called the Goldilocks of models, and previously

**[00:09:49 – 00:09:53]** for Nemotron 3 we had released a Lightning, a Super and an

**[00:09:53 – 00:10:00]** Ultra model, and then in September we released a Lightning 3.5,

**[00:10:00 – 00:10:03]** which took all of our knowledge from our Ultra model and kind

**[00:10:03 – 00:10:06]** of put it back into this little Lightning format of a 30 billion,

**[00:10:06 – 00:10:08]** all of these, by the by the way, if you're familiar with

**[00:10:08 – 00:10:13]** a mixture of experts, they're also hybrid technology, so state-based

**[00:10:13 – 00:10:17]** model plus a transformer on a mixture of experts architecture.

**[00:10:17 – 00:10:22]** We have a 30 billion parameter model, a 120 billion parameter

**[00:10:23 – 00:10:27]** model for the Super, and then a 550 billion parameter model for Ultra.

**[00:10:27 – 00:10:33]** But what we did last week was that we released Nemotron 3.5 Super VL.

**[00:10:33 – 00:10:37]** So we took the same architecture of our Super model, we took

**[00:10:38 – 00:10:40]** some bugs that we opened up, again we treat these like libraries,

**[00:10:40 – 00:10:44]** we have a roadmap, we're releasing versions all the time, new updates.

**[00:10:45 – 00:10:47]** And so we took knowledge from our Ultra model, we put it

**[00:10:47 – 00:10:50]** into our Super, we took knowledge from our Omni model, we put

**[00:10:50 – 00:10:51]** it into the Super.

**[00:10:51 – 00:10:56]** So now Super 3.5 understands images and videos.

**[00:10:56 – 00:11:00]** Now what we did with the Super model was we compressed it.

**[00:11:00 – 00:11:03]** And then we quantized it down to a two-bit model, which

**[00:11:03 – 00:11:04]** resulted in the Bolt.

**[00:11:04 – 00:11:10]** And so Bolt is sitting at, I think, 24 gigabytes.

**[00:11:10 – 00:11:14]** So it's great for your RTX Spark to fit on there with

**[00:11:14 – 00:11:16]** the intelligence of the supermodel.

**[00:11:16 – 00:11:18]** So we're really excited about that.

**[00:11:18 – 00:11:21]** Again, as Justin said, we have open datasets.

**[00:11:21 – 00:11:24]** We also have something called Nemo UserSim, which I'll talk

**[00:11:24 – 00:11:27]** a little bit about, to be able to simulate populations

**[00:11:28 – 00:11:29]** with our personas dataset.

**[00:11:29 – 00:11:32]** And then, of course, Nemo RL and our GIM environments

**[00:11:32 – 00:11:34]** for evaluation and verifiers.

**[00:11:34 – 00:11:37]** And then open research, if you watch any papers that

**[00:11:37 – 00:11:39]** come out and models that we publish at Hugging Face, you

**[00:11:39 – 00:11:44]** can find those under Nemotron-labs to see some of those models, the

**[00:11:44 – 00:11:48]** research research that ultimately goes into our Nemotron models.

**[00:11:48 – 00:11:52]** Now, what we did at the end of training is that we really focused

**[00:11:52 – 00:11:56]** on our multi-token prediction and the Lilycore version of that.

**[00:11:57 – 00:12:01]** And so we were able to get 5x faster output speed for

**[00:12:01 – 00:12:03]** Nemotron Super, if you can even see here compared to

**[00:12:03 – 00:12:06]** our Super, our Nemotron 3 Super.

**[00:12:06 – 00:12:09]** Our Nemotron 3.5 is almost five times faster, and so

**[00:12:09 – 00:12:11]** we're really excited, especially about, you can see how much

**[00:12:11 – 00:12:15]** faster it is among all these cohorts and what that means is when

**[00:12:15 – 00:12:20]** you are, again, doing a multi-turn, multi-step, long-running agents,

**[00:12:20 – 00:12:23]** that the amount it takes to get a task done, to do a tool

**[00:12:23 – 00:12:28]** use, to make that call, to reason about something, it all adds up.

**[00:12:28 – 00:12:31]** Now, we need to continuously improve these models.

**[00:12:31 – 00:12:35]** Again, since we are completely open and out of the box, doesn't

**[00:12:35 – 00:12:36]** necessarily work in a domain.

**[00:12:36 – 00:12:38]** Many people know that.

**[00:12:38 – 00:12:41]** And so we provide Nemo RL.

**[00:12:41 – 00:12:46]** We provide the libraries for synthetic data generation, being

**[00:12:46 – 00:12:51]** able to synthesize of that and be able to do anonymization of that.

**[00:12:51 – 00:12:54]** And then we also have these open environments.

**[00:12:54 – 00:12:57]** And so, again, we put out all of our RL environments

**[00:12:57 – 00:13:01]** for Nemotron, which is very unique for open weights, but

**[00:13:01 – 00:13:04]** we consider ourselves open source, because again, not only the data

**[00:13:04 – 00:13:06]** sets, but these RL environments.

**[00:13:06 – 00:13:09]** But then we're working with our partners to be able to

**[00:13:09 – 00:13:10]** create RL environments for them.

**[00:13:10 – 00:13:14]** And I'm going to get to that in a second about how we're helping our

**[00:13:14 – 00:13:17]** partners will be able to put things into GIM to be able to create these

**[00:13:17 – 00:13:19]** environments and create verifiers.

**[00:13:19 – 00:13:25]** But you do this loop, and you can use your favorite agent

**[00:13:25 – 00:13:28]** like Codex to be able to drive the automation of this RL loop.

**[00:13:28 – 00:13:30]** And so when we worked with our partners, my favorite

**[00:13:30 – 00:13:34]** part about Nemotron is that it's not just one domain.

**[00:13:34 – 00:13:37]** This is across all kinds of domains from document intelligence

**[00:13:37 – 00:13:42]** and cybersecurity, finance, scientific, material science,

**[00:13:42 – 00:13:43]** and even medical insurance.

**[00:13:43 – 00:13:46]** And so you can see here that all these partners were able

**[00:13:46 – 00:13:49]** to greatly customize Nemotron.

**[00:13:49 – 00:13:51]** And you might say, okay, well, it's just like a little bit

**[00:13:51 – 00:13:52]** better than a frontier model.

**[00:13:52 – 00:13:54]** But you have to remember, number one, that's pretty

**[00:13:54 – 00:13:59]** awesome, that you're able to have your own beyond frontier

**[00:13:59 – 00:14:01]** level model for your domain.

**[00:14:01 – 00:14:05]** But then you're also keeping your data, all of your data,

**[00:14:05 – 00:14:07]** all of your workflow, all of your knowledge you are now able

**[00:14:07 – 00:14:10]** to keep and continually improve that, and so all of these companies

**[00:14:10 – 00:14:14]** are on their way to maintain control of their data, the privacy

**[00:14:14 – 00:14:18]** of their data, and then become experts within and maintain staying

**[00:14:18 – 00:14:20]** experts within their domain.

**[00:14:20 – 00:14:23]** And beyond customization, obviously, we showed the

**[00:14:23 – 00:14:27]** difference of throughput with SuperVL and 5x faster.

**[00:14:27 – 00:14:32]** So you get 50% lower inference costs, you get higher triage

**[00:14:32 – 00:14:36]** accuracy in cybersecurity, you get higher token throughput

**[00:14:36 – 00:14:38]** for computer use.

**[00:14:38 – 00:14:40]** Again, 60% fewer output tokens.

**[00:14:40 – 00:14:42]** You can just see all these different business values across

**[00:14:42 – 00:14:44]** every single different domain.

**[00:14:44 – 00:14:48]** And we have a lot of post-training partners that we work with

**[00:14:48 – 00:14:50]** because we have a lot of customers who say, you know what, I

**[00:14:50 – 00:14:53]** actually don't have the skills in-house to be able to create

**[00:14:53 – 00:14:56]** these environments or create the verifiers or evaluation benchmarks.

**[00:14:56 – 00:14:59]** And so we work with the post-training ecosystem

**[00:14:59 – 00:15:03]** below to be able to do some matchmaking amongst customers

**[00:15:03 – 00:15:06]** and post-training partners.

**[00:15:06 – 00:15:10]** Okay, Sovereign AI, Localized AI.

**[00:15:10 – 00:15:12]** We've been working, we've been releasing these, we've

**[00:15:12 – 00:15:15]** been synthetically generating data sets, making sort of digital

**[00:15:15 – 00:15:21]** twins based on census data around the world for different countries.

**[00:15:21 – 00:15:23]** And so you can see here the ones, it's a little bit of

**[00:15:23 – 00:15:27]** an eye chart, but the ones that are live are circled in green.

**[00:15:27 – 00:15:30]** We have new emerging cohorts in different regions like

**[00:15:30 – 00:15:33]** Japan and UK and some other ones in the pipeline.

**[00:15:33 – 00:15:38]** There's actually a model builders meet up tomorrow about being

**[00:15:38 – 00:15:42]** able to collect data regionally in Europe so we can improve Nemotron.

**[00:15:43 – 00:15:46]** Basically, you want to take the nuance of your local data,

**[00:15:46 – 00:15:52]** customs, behaviors, and special data like your health, insurance,

**[00:15:52 – 00:15:54]** finance, government documents.

**[00:15:54 – 00:15:57]** You need to be able to take those documents, keep them locally, and

**[00:15:57 – 00:15:59]** then we put them in data designer.

**[00:15:59 – 00:16:00]** We create more synthetic data.

**[00:16:01 – 00:16:05]** We do personas to be able to do, again, digital twin-of-year,

**[00:16:06 – 00:16:09]** of your census data, and then we're able to use NeMo user

**[00:16:09 – 00:16:13]** sim to be able to simulate an outcome based on your population

**[00:16:13 – 00:16:16]** and the sort of definition of your population and simulate

**[00:16:16 – 00:16:20]** this data to see how the model might perform for your region.

**[00:16:20 – 00:16:21]** So it's really exciting.

**[00:16:21 – 00:16:25]** We've released billions of tokens already with personas,

**[00:16:25 – 00:16:29]** and again we're really excited about NeMo user sim, which

**[00:16:29 – 00:16:31]** is brand, that's brand new.

**[00:16:31 – 00:16:35]** We worked with Siemens for their EDA chip design to be

**[00:16:35 – 00:16:39]** able to, again, take all of this through their 3D integrated

**[00:16:39 – 00:16:42]** circuit chip design, and the result is, again, we help

**[00:16:42 – 00:16:45]** them create these environments, we help them post-train, put

**[00:16:45 – 00:16:50]** that out into a gym, do Nemo RL, and the output was faster

**[00:16:50 – 00:16:52]** configuration for these engineers.

**[00:16:52 – 00:16:56]** This is a very engineering intense manual, so then they

**[00:16:56 – 00:17:00]** had better design space, coverage, and less engineering effort

**[00:17:01 – 00:17:03]** at the end of the day.

**[00:17:04 – 00:17:07]** But all of this needs to be grounded in

**[00:17:07 – 00:17:08]** foundational information.

**[00:17:08 – 00:17:12]** Yeah, so the next body of work that typically companies

**[00:17:12 – 00:17:16]** need to do is they've got to connect into their unstructured

**[00:17:16 – 00:17:18]** and structured documents.

**[00:17:18 – 00:17:22]** And so what we announced here is Agent Knowledge Foundation.

**[00:17:22 – 00:17:26]** This includes, again, sort of families of models and libraries to

**[00:17:26 – 00:17:30]** be able to continuously ingest both structured and unstructured data,

**[00:17:30 – 00:17:34]** to be able to ask what-if scenarios and be able to look out into

**[00:17:34 – 00:17:38]** the, ask questions like predictive questions into the future.

**[00:17:39 – 00:17:42]** We will go into, I guess, each one of these.

**[00:17:42 – 00:17:46]** You know, on the start of this, they're all at the top

**[00:17:46 – 00:17:47]** of the leaderboard.

**[00:17:47 – 00:17:50]** Yeah, they're all top of the leaderboard in their

**[00:17:50 – 00:17:50]** respective categories.

**[00:17:50 – 00:17:55]** I think Jensen touched on that in the keynote, if everybody saw it.

**[00:17:56 – 00:17:58]** On the unstructured side, it usually starts with

**[00:17:58 – 00:18:02]** embedding, re-ranking, document parsing, vector search.

**[00:18:02 – 00:18:05]** So QVS is a vector search library.

**[00:18:05 – 00:18:10]** All of these other ones are open models that we put out

**[00:18:10 – 00:18:15]** there for helping do document understanding and extraction.

**[00:18:15 – 00:18:19]** So you can do semantic search, unstructured data, and again

**[00:18:19 – 00:18:21]** these, these are winning a range of leaderboards.

**[00:18:21 – 00:18:25]** They're all foundational for you guys to build from and

**[00:18:25 – 00:18:28]** bring your unstructured data, which is effectively, I think,

**[00:18:28 – 00:18:31]** 70% of enterprise data still sits in unstructured data

**[00:18:31 – 00:18:32]** across your enterprise.

**[00:18:32 – 00:18:36]** So whether it's, you know, sales sheets on your business,

**[00:18:36 – 00:18:42]** contracts, you know, anything else specific to your industry,

**[00:18:42 – 00:18:45]** like research papers, like all of that information can

**[00:18:45 – 00:18:48]** come in through unstructured data.

**[00:18:48 – 00:18:53]** Do you wanna, I guess, talk a bit about?

**[00:18:53 – 00:18:57]** Oh, yeah, so embedding lowers agentic costs.

**[00:18:57 – 00:19:01]** So basically, when you have the right information surfaced first,

**[00:19:01 – 00:19:04]** then you don't have to continually retrieve information or get

**[00:19:04 – 00:19:05]** the wrong information at the back.

**[00:19:05 – 00:19:08]** So you kind of cut down the number of loops that are happening

**[00:19:08 – 00:19:09]** during that agentic task.

**[00:19:09 – 00:19:13]** And so we do that not only by first the embedding model,

**[00:19:13 – 00:19:15]** but a lot of companies might have started with an embedding

**[00:19:15 – 00:19:18]** model already, and they don't want to have to re-embed.

**[00:19:18 – 00:19:21]** And so being able to add a re-ranking model at the top

**[00:19:21 – 00:19:23]** of that, so you're surfacing and bubbling the most important

**[00:19:23 – 00:19:27]** information up, and people say, well, maybe that re-ranking

**[00:19:27 – 00:19:30]** model, if it's still in LLM, I can't take that latency.

**[00:19:30 – 00:19:33]** You'd be surprised because the latency is still encapsulated

**[00:19:33 – 00:19:34]** in the large language model.

**[00:19:34 – 00:19:37]** And so being able to, you actually lower the cost of

**[00:19:37 – 00:19:41]** the total system because you're retrieving information, the

**[00:19:41 – 00:19:44]** correct information faster, and that's what the accuracy

**[00:19:44 – 00:19:46]** of our models provides.

**[00:19:46 – 00:19:50]** And so the next family of models, the other big body

**[00:19:50 – 00:19:53]** of information in the world's enterprises, I'll call it the high

**[00:19:53 – 00:19:57]** accuracy data of your business, all sits in structured data.

**[00:19:57 – 00:20:01]** Historically, to do predictive analytics on structured data,

**[00:20:01 – 00:20:05]** you'd have to train an XGBoost model on a specific set of features

**[00:20:06 – 00:20:07]** and then project into the future.

**[00:20:07 – 00:20:09]** And you'd have to have data scientists ask these, you

**[00:20:09 – 00:20:11]** know, ask a very specific question; it would take them

**[00:20:12 – 00:20:14]** some time to train those models.

**[00:20:14 – 00:20:17]** What's nice is, with Kumo, which is a new family of open

**[00:20:17 – 00:20:21]** tabular foundation models, this model allows you to ask

**[00:20:21 – 00:20:26]** any what-if questions across any structured data, so you

**[00:20:26 – 00:20:28]** don't have to pre-train all the dimensions, and you can

**[00:20:28 – 00:20:32]** start to ask, you know, what if I have a supply shortage on memory?

**[00:20:32 – 00:20:36]** Like, you know, what would happen to, like, my build plan?

**[00:20:36 – 00:20:39]** And you can start to look at forecasts into the future.

**[00:20:39 – 00:20:43]** So this is a very powerful set of new models.

**[00:20:43 – 00:20:47]** There's tabular models, there's forecast models, there's anomaly

**[00:20:47 – 00:20:52]** models, and so we think this is just a huge, I guess, untapped

**[00:20:52 – 00:20:56]** area for understanding structured data and letting agents ask

**[00:20:56 – 00:21:00]** questions and do scenario planning into the structured data

**[00:21:00 – 00:21:03]** that exists within enterprises.

**[00:21:04 – 00:21:08]** Kumo, like many of our other models, really is pushing

**[00:21:08 – 00:21:12]** the frontier of intelligence in this category of models.

**[00:21:12 – 00:21:16]** Again, we put the models out there and the techniques out there; all

**[00:21:16 – 00:21:21]** this research is out there for you to build from and to build with.

**[00:21:21 – 00:21:24]** And so we're really excited to be able to, I guess, share

**[00:21:24 – 00:21:26]** this new family of models.

**[00:21:26 – 00:21:30]** We're seeing this adopted in platform companies like Snowflake.

**[00:21:30 – 00:21:35]** They can put it into Snowflake ML to let enterprises ask

**[00:21:35 – 00:21:40]** these what-if scenarios of the business data that they have.

**[00:21:40 – 00:21:42]** OK, so now you have your agent, but how do you optimize it?

**[00:21:42 – 00:21:48]** That can be a daunting task, even for coding agents or humans alike.

**[00:21:48 – 00:21:52]** And so what we've done is that we've worked with a lot

**[00:21:52 – 00:21:55]** of the open agent harnesses, and we've integrated some libraries.

**[00:21:55 – 00:21:58]** And so we can consider these some of the tools.

**[00:21:58 – 00:22:02]** So, I'm going to go through these three green boxes in

**[00:22:02 – 00:22:04]** this agent optimization loop.

**[00:22:04 – 00:22:06]** First, what we've done is we've integrated NeMo Relay

**[00:22:06 – 00:22:08]** into a lot of these open harnesses.

**[00:22:08 – 00:22:12]** And what Relay does is able to sort of take the logging or agentic

**[00:22:12 – 00:22:16]** traces and be able to put them out into an open format; you sort of

**[00:22:16 – 00:22:19]** dump them somewhere, but then you have to go do something with that.

**[00:22:19 – 00:22:21]** And I'll talk about that in a second.

**[00:22:21 – 00:22:25]** The thing is, these agentic traces are coming out by the millions.

**[00:22:25 – 00:22:28]** Think of all, again, Justin talked about the five things

**[00:22:28 – 00:22:32]** that are happening around an agent, and then so the harness

**[00:22:32 – 00:22:35]** is controlling all of that because it's a new computer architecture.

**[00:22:35 – 00:22:37]** And so you have all these traces happening across the

**[00:22:37 – 00:22:41]** thought against the tool call chain, the memory, the compaction,

**[00:22:41 – 00:22:45]** and so how do I even, you've got to capture that first, and then once

**[00:22:45 – 00:22:48]** you capture it, you're like, holy cow, I've got to go through it.

**[00:22:48 – 00:22:51]** And so what we've done is that we've released a repository,

**[00:22:51 – 00:22:56]** an open repository called NeMo Compass, and this allows

**[00:22:56 – 00:23:00]** you to be able to go through and look for signals within

**[00:23:00 – 00:23:02]** your agentic trace data.

**[00:23:02 – 00:23:05]** And you would say, well, okay, couldn't I just point a coding

**[00:23:05 – 00:23:09]** agent at the, Is there an agent at the ATIF or ATOF

**[00:23:09 – 00:23:11]** that really log data and let it go?

**[00:23:11 – 00:23:13]** And the answer is yes, but the problem is, now

**[00:23:13 – 00:23:15]** an agent can go deep.

**[00:23:15 – 00:23:18]** The problem is the volume of data now that is coming

**[00:23:18 – 00:23:21]** out on these agentic traces is, it can be extremely costly.

**[00:23:21 – 00:23:25]** And so, you know, we're being able to, we have skills that

**[00:23:25 – 00:23:27]** you can provide your agent with to be able to go through

**[00:23:28 – 00:23:32]** these large volumes of agentic traces for signals of being able.

**[00:23:32 – 00:23:36]** We're able to optimize web for quality, for safety, and then

**[00:23:36 – 00:23:41]** be able to sort of make suggestions on what you should do to improve.

**[00:23:41 – 00:23:44]** And then once you do that, you've gone through all this

**[00:23:44 – 00:23:46]** and you're like, okay, I can improve this, but then I've

**[00:23:46 – 00:23:52]** got to go create an environment or a benchmark or evaluation.

**[00:23:52 – 00:23:55]** And I have to say that, honestly, for every partner that I work

**[00:23:55 – 00:23:59]** with, evaluation and benchmarking is the hardest thing.

**[00:23:59 – 00:24:02]** This is where most people get stuck, being able to create that

**[00:24:02 – 00:24:05]** test case and that verifier to be able to say, how well is my agent

**[00:24:06 – 00:24:07]** doing across all these scenarios?

**[00:24:07 – 00:24:13]** And so we created NeMo Evaluator, again, an open source repository

**[00:24:13 – 00:24:18]** to be able to go out and really be able to look at these traces,

**[00:24:19 – 00:24:22]** improve, go through them, sorry, go through your evals,

**[00:24:22 – 00:24:26]** go through them and say, you know, you're missing, and score them and

**[00:24:26 – 00:24:30]** say you're missing some evaluations across these areas that we saw.

**[00:24:30 – 00:24:32]** You're doing good here. So it's scoring and then ranking

**[00:24:32 – 00:24:37]** of evaluations to either go create or go improve.

**[00:24:37 – 00:24:41]** And so all of these areas that we did here for this

**[00:24:41 – 00:24:44]** agent improvement task is what we did with Dassault Systems.

**[00:24:44 – 00:24:48]** And so if you saw in Jensen's keynote demo earlier today,

**[00:24:48 – 00:24:51]** we worked really closely with Dassault Systems as a design

**[00:24:51 – 00:24:55]** partner for their Leo agent.

**[00:24:55 – 00:25:01]** And so being able to take, take the super 3.5 VL, they

**[00:25:01 – 00:25:05]** fine-tuned it, they put it into their agent stack, they

**[00:25:05 – 00:25:08]** use relay, they dump the traces, and you would be surprised because

**[00:25:08 – 00:25:11]** I think they tried to do it by hand at first, and so they were this

**[00:25:11 – 00:25:15]** really great co-design partner with Compass to be able to say, okay,

**[00:25:15 – 00:25:19]** this really helped us to be able to figure out what the right signals

**[00:25:19 – 00:25:23]** were, go create the evaluation, and then totally improve our agent.

**[00:25:23 – 00:25:26]** This, if you saw, the Dassault Systems was working with

**[00:25:26 – 00:25:28]** Volkswagen at the time.

**[00:25:28 – 00:25:32]** They were able to get a 20% higher door design quality,

**[00:25:33 – 00:25:36]** gain just access to broader design.

**[00:25:36 – 00:25:39]** So they were able to simulate different designs overnight.

**[00:25:39 – 00:25:41]** So they're taking this whole process, this manual process,

**[00:25:41 – 00:25:43]** from weeks to hours.

**[00:25:43 – 00:25:47]** And then again, when you have your proprietary tool call,

**[00:25:47 – 00:25:48]** when you're able to train on your.

**[00:25:48 – 00:25:53]** Now you don't have this guessing game; you're able to improve

**[00:25:53 – 00:25:56]** the type of tool calling on your proprietary tools, and

**[00:25:56 – 00:25:59]** then also calling accelerated tools like Katia.

**[00:25:59 – 00:26:03]** So they were a really great design partner for this virtual companion.

**[00:26:04 – 00:26:06]** Yeah, so the next, I guess, set of challenges that come

**[00:26:06 – 00:26:10]** up in enterprises, that the adoption of AI has been so fast

**[00:26:10 – 00:26:13]** that you have all of these agents running within your organization.

**[00:26:13 – 00:26:16]** A lot of the conversation up to this point has been around,

**[00:26:16 – 00:26:19]** honestly, model alignment, like, how do I make sure that the agent

**[00:26:19 – 00:26:21]** behaves well in any environment?

**[00:26:21 – 00:26:25]** The challenge is you can't train a model to know all the local rules

**[00:26:25 – 00:26:29]** and regulations and your business context and really understand how

**[00:26:29 – 00:26:30]** to apply it in your environment.

**[00:26:30 – 00:26:33]** So what you need to do is you need to have deterministic

**[00:26:33 – 00:26:36]** controls that you can ensure can enforce what an agent

**[00:26:36 – 00:26:41]** can do in your business, where can it take data, what can it access.

**[00:26:41 – 00:26:44]** You should treat the agents just like you treat your employees.

**[00:26:44 – 00:26:46]** You should give them least privilege to do the job that

**[00:26:46 – 00:26:48]** you're asking them to do.

**[00:26:48 – 00:26:51]** And if you don't, I think about a year ago, we started

**[00:26:51 – 00:26:54]** seeing this problem where you'd ask an agent to do something on

**[00:26:54 – 00:26:58]** your behalf, it would misunderstand what you're asking it to do,

**[00:26:58 – 00:27:02]** or it would be blocked from doing the thing that you're asking it not

**[00:27:02 – 00:27:06]** to do, and it might find another tool to achieve the objective.

**[00:27:06 – 00:27:09]** And so that can turn into data leakage problems, that

**[00:27:09 – 00:27:15]** can turn into APIs, keys going places you don't want, and

**[00:27:15 – 00:27:18]** obviously all of us want to keep our knowledge, our know-how

**[00:27:18 – 00:27:20]** private to our organization.

**[00:27:21 – 00:27:24]** So what we needed to do was build a new control layer for agents.

**[00:27:24 – 00:27:28]** Traditional sandboxes just weren't secure enough.

**[00:27:28 – 00:27:32]** And this is why, frankly, when you see leaks from labs

**[00:27:32 – 00:27:36]** in these long-running agentic tasks, the leak is happening

**[00:27:36 – 00:27:40]** because traditional sandboxing security wasn't good enough.

**[00:27:40 – 00:27:42]** So what we did is we built a security layer that is outside

**[00:27:42 – 00:27:45]** the reach of the agents.

**[00:27:46 – 00:27:49]** This control layer that we call OpenShell runs in the

**[00:27:49 – 00:27:53]** kernel of the operating system, and it enforces network access,

**[00:27:53 – 00:27:58]** file system control, process level control, what tool can

**[00:27:58 – 00:28:02]** the agent use, and where can it go to talk to an LLM in a very

**[00:28:03 – 00:28:04]** strict way, policy-enforced way.

**[00:28:04 – 00:28:10]** It allows security teams to author the policy, what can the agent do.

**[00:28:10 – 00:28:14]** We also have tools as part of OpenShell where you can ask

**[00:28:14 – 00:28:20]** what-ifs, like, if I change this policy, you know, can the agent

**[00:28:20 – 00:28:21]** get access to the internet somehow?

**[00:28:21 – 00:28:24]** Because as you start to run fleets of agents, you can

**[00:28:24 – 00:28:27]** imagine if agents can talk to each other and they have

**[00:28:27 – 00:28:31]** different scopes, and one of them can get to the internet, obviously

**[00:28:31 – 00:28:32]** they can all get to the internet.

**[00:28:32 – 00:28:35]** And so you've gotta be able to do mathematical verification

**[00:28:35 – 00:28:38]** of what can the agents do before you let them run.

**[00:28:38 – 00:28:44]** We offer this open source as part of OpenShell, Apache 2 license,

**[00:28:44 – 00:28:46]** you know, broadly supported.

**[00:28:46 – 00:28:48]** Jensen talked about this a bit.

**[00:28:48 – 00:28:51]** The second layer of defense we call BlueField.

**[00:28:51 – 00:28:55]** You can offload and enforce things in silicon.

**[00:28:55 – 00:28:58]** You can have a secondary monitor so the agents don't know that

**[00:28:58 – 00:28:59]** they're being watched.

**[00:28:59 – 00:29:03]** Where this becomes super critical is if you're using, I'll say,

**[00:29:03 – 00:29:06]** models that haven't been through any alignment, or you're using

**[00:29:06 – 00:29:10]** models where you've taken off the guardrails for offensive red

**[00:29:10 – 00:29:14]** teaming, you want to make sure you don't accidentally attack somebody.

**[00:29:14 – 00:29:17]** And so you can have this second layer of defense that's watching

**[00:29:17 – 00:29:20]** the agent behavior all the time, and you can have classifiers

**[00:29:20 – 00:29:22]** so that you can turn.

**[00:29:23 – 00:29:26]** So, you can imagine, so now I've got thousands of agents

**[00:29:26 – 00:29:29]** across NVIDIA, and the next big problem I'm asking myself is, well,

**[00:29:29 – 00:29:30]** how do I author these policies?

**[00:29:30 – 00:29:32]** How do I sign these policies?

**[00:29:32 – 00:29:35]** How do I push them to all the environments where the agents run?

**[00:29:35 – 00:29:37]** And that's where we're working with our ecosystem

**[00:29:37 – 00:29:41]** and partners like ServiceNow.

**[00:30:26 – 00:30:31]** That was the first time I heard the audio, but we're working

**[00:30:31 – 00:30:34]** with, I'd say, hundreds of partners across the security ecosystem

**[00:30:34 – 00:30:38]** to try and plumb OpenShell into all these different control points.

**[00:30:38 – 00:30:40]** We are not the agent governance company.

**[00:30:41 – 00:30:43]** We're not the human governance company, but we understand

**[00:30:43 – 00:30:47]** the controls and how to build data processing, data acceleration tools

**[00:30:47 – 00:30:49]** that plumb into the ecosystem.

**[00:30:49 – 00:30:52]** And so this adoption now of OpenShell is happening super

**[00:30:52 – 00:30:56]** quickly through all, I'd say, critical infrastructure.

**[00:30:56 – 00:30:58]** And I also saw a little peek of SwitchArt in there,

**[00:30:58 – 00:30:59]** too, in that demo.

**[00:30:59 – 00:31:01]** Yeah, nice.

**[00:31:01 – 00:31:05]** Came back to it, and the other thing that we announced today

**[00:31:05 – 00:31:08]** is this platform we call OpenShell Reference Architecture.

**[00:31:08 – 00:31:12]** It really uses two types of systems for running models

**[00:31:12 – 00:31:14]** and running agents.

**[00:31:14 – 00:31:17]** On the one hand, running the models, like, you need big

**[00:31:17 – 00:31:22]** GPU systems to be able to run these large models, potentially

**[00:31:22 – 00:31:24]** across rack-scale infrastructure.

**[00:31:24 – 00:31:27]** Jensen talked a lot about how we do extreme co-design of that.

**[00:31:27 – 00:31:32]** On the other side, the harnesses want to run on CPU systems.

**[00:31:32 – 00:31:36]** Our Vera CPU system is a great CPU system, but I'll say all

**[00:31:36 – 00:31:38]** the work that we're doing with OpenShell, we make sure

**[00:31:38 – 00:31:42]** runs on x86 or ARM so that it will run in anybody's infrastructure

**[00:31:43 – 00:31:43]** that they have today.

**[00:31:43 – 00:31:46]** It's not, nothing there is, like, tied to our hardware.

**[00:31:46 – 00:31:50]** We're just doing it really as stewards of AI.

**[00:31:50 – 00:31:53]** We believe AI safety and AI security is obviously critical

**[00:31:53 – 00:31:56]** for the advancement of the industry and to allow enterprises

**[00:31:56 – 00:32:01]** to write the policies that they need in their environment.

**[00:32:01 – 00:32:06]** So I think that, just to kind of recap a little bit from echoing

**[00:32:06 – 00:32:08]** what Jensen said this morning and what Justin started with

**[00:32:08 – 00:32:11]** about these five components of an agent, and obviously there's more.

**[00:32:11 – 00:32:15]** I mean, we can get down to the details of control loops

**[00:32:15 – 00:32:17]** and memory compaction stuff, but let's just keep it high level here.

**[00:32:17 – 00:32:21]** You have the models, you have the harness, you have the runtime,

**[00:32:21 – 00:32:25]** you have a knowledge foundation, and you have access to tools.

**[00:32:25 – 00:32:28]** And so when we kind of break it apart in that way, we also

**[00:32:28 – 00:32:31]** have all of the libraries, models, and tools to kind

**[00:32:31 – 00:32:33]** of plug in and make agents better.

**[00:32:33 – 00:32:36]** And that's kind of what we refer to as the NVIDIA Agent Toolkit.

**[00:32:36 – 00:32:40]** And so if you go, the NVIDIA Agent Toolkit isn't something

**[00:32:40 – 00:32:42]** you go and download or pip install.

**[00:32:42 – 00:32:44]** It's all of these different libraries that you can

**[00:32:44 – 00:32:47]** pick and choose which ones you need for task-specific

**[00:32:47 – 00:32:48]** improvements of your agents.

**[00:32:48 – 00:32:51]** And that's also when we say, when we break it apart this

**[00:32:51 – 00:32:55]** way in these components, this is how we also engage with

**[00:32:55 – 00:33:00]** our ecosystem, which we're going to get to after this video.

**[00:33:00 – 00:33:04]** The most intelligent scientist still needs a laboratory.

**[00:33:04 – 00:33:08]** The best engineer still needs simulators and solvers.

**[00:33:08 – 00:33:11]** An AI agent is no different.

**[00:33:12 – 00:33:15]** An agent brings together five building blocks.

**[00:33:15 – 00:33:18]** The model perceives, reasons, and plans.

**[00:33:18 – 00:33:20]** The harness coordinates the work.

**[00:33:21 – 00:33:24]** Knowledge Foundation gives the agent context from your company's

**[00:33:24 – 00:33:26]** structured and unstructured data.

**[00:33:27 – 00:33:31]** Tools let the agent simulate, experiment, and act.

**[00:33:31 – 00:33:35]** NVIDIA OpenShell controls what agents can access and execute.

**[00:33:35 – 00:33:39]** Start with a frontier closed model or customize an open model.

**[00:33:39 – 00:33:42]** With NVIDIA's Agent Toolkit, any company can turn its

**[00:33:42 – 00:33:47]** knowledge and expertise into a customized super agent.

**[00:33:47 – 00:33:49]** Let's look at three examples.

**[00:33:49 – 00:33:53]** For Lyla Sciences, the goal is to design a therapy

**[00:33:53 – 00:33:55]** for B-cell lymphoma.

**[00:33:55 – 00:33:59]** The Lyla Iris agent, built on NVIDIA NEMOTRON, turns that

**[00:33:59 – 00:34:02]** goal into an experimental plan.

**[00:34:02 – 00:34:06]** In the computational dry lab, Lila agents use NVIDIA BioNeMo

**[00:34:06 – 00:34:10]** Agent Toolkit to call models for protein design and RNA

**[00:34:10 – 00:34:14]** optimization, running millions of virtual experiments.

**[00:34:14 – 00:34:17]** The most promising candidates move into Lila's AI science

**[00:34:18 – 00:34:21]** factory, where autonomous robots and instruments, developed

**[00:34:21 – 00:34:25]** with NVIDIA Omniverse and Cosmos, make and test them.

**[00:34:25 – 00:34:30]** The results return to IRIS, shaping the next designs and experiments.

**[00:34:30 – 00:34:32]** The loop continues.

**[00:34:32 – 00:34:36]** At Siemens Electronic Factory Erlangen, the goal is to bring

**[00:34:36 – 00:34:38]** new products into production.

**[00:34:38 – 00:34:43]** Process preparation ex-agents use NVIDIA Cosmos to generate virtual

**[00:34:43 – 00:34:47]** prototypes and synthetic data, then train and validate inspection

**[00:34:47 – 00:34:50]** models before production starts.

**[00:34:50 – 00:34:53]** Intelligence center ex-agents, built with NVIDIA's factory

**[00:34:53 – 00:34:58]** operations blueprint, use orders, supplies, and line setups

**[00:34:58 – 00:35:00]** to recommend production plans.

**[00:35:00 – 00:35:05]** Digital Twin Composer uses NVIDIA Omniverse libraries to validate

**[00:35:05 – 00:35:09]** those plans with physics simulation and real-time factory data before

**[00:35:09 – 00:35:13]** changes reach the factory floor.

**[00:35:13 – 00:35:16]** With the Dassault Systems platform, an engineer needs

**[00:35:16 – 00:35:20]** to adapt a Volkswagen Golf door design for the Tiguan.

**[00:35:20 – 00:35:23]** Aura and Leo use NVIDIA Nematron to reason over

**[00:35:23 – 00:35:25]** engineering requirements.

**[00:35:25 – 00:35:29]** The harness coordinates the specialized sub-agents and

**[00:35:29 – 00:35:33]** preserves context across planning, design, and simulation.

**[00:35:33 – 00:35:36]** Aura uses Inovia to plan the work around the vehicle's

**[00:35:36 – 00:35:41]** dimensions, weight limits, and other engineering requirements.

**[00:35:41 – 00:35:44]** Leo Design Agent selects a compatible golf door design

**[00:35:45 – 00:35:48]** and uses CATIA to reshape it for the Tiguan.

**[00:35:48 – 00:35:52]** LEO Simulation Agent uses Simulia, accelerated by NVIDIA CUDA

**[00:35:52 – 00:35:57]** X libraries, to test the variants, revealing a vibration problem.

**[00:35:57 – 00:36:00]** The results return to the design agent for refinement.

**[00:36:00 – 00:36:05]** The engineer reviews the refined designs and selects the final one.

**[00:36:06 – 00:36:07]** Different goals.

**[00:36:07 – 00:36:09]** Different expertise.

**[00:36:09 – 00:36:11]** The same building blocks.

**[00:36:11 – 00:36:15]** Open Models, CUDA X Accelerated Tools, and the OpenShell Secure

**[00:36:15 – 00:36:21]** Runtime, brought together with the NVIDIA Agent Toolkit.

**[00:36:21 – 00:36:25]** You know, I was at the Siemens manufacturing plant in Erlangen on

**[00:36:25 – 00:36:28]** Monday, and what wasn't mentioned in that video is that they actually

**[00:36:28 – 00:36:32]** had a Nemotron in production for about two months now, doing

**[00:36:32 – 00:36:34]** the planning for their service mount technology planning, because

**[00:36:34 – 00:36:37]** they have about 2,000 SKUs that they always have to maintain and

**[00:36:37 – 00:36:39]** get new orders every single day.

**[00:36:39 – 00:36:41]** They have to reconfigure machines, and so I just thought that

**[00:36:41 – 00:36:43]** was a really cool...

**[00:36:43 – 00:36:45]** This is a use case because what it did is it took the

**[00:36:45 – 00:36:49]** planning of these machines from four hours down to, every day,

**[00:36:49 – 00:36:52]** four hour planning down to about under 30 minutes, and so again,

**[00:36:52 – 00:36:55]** this is just, and they get to keep their data, they get to keep their

**[00:36:55 – 00:36:59]** IP and knowledge of that workflow of reconfiguring their machines.

**[00:36:59 – 00:37:04]** So, again, if you don't understand or have heard of our go-to-market

**[00:37:04 – 00:37:08]** strategy for NVIDIA, we go through our partners, we go

**[00:37:08 – 00:37:09]** through our ecosystem.

**[00:37:09 – 00:37:11]** Of course, we have these open libraries, and you can get

**[00:37:11 – 00:37:14]** them directly, but we love to go through our partners,

**[00:37:14 – 00:37:15]** love to work with our partners.

**[00:37:15 – 00:37:19]** So again, there's all kinds of open models, people who

**[00:37:19 – 00:37:21]** are fine-tuning on our open models, and they can't do that, they

**[00:37:22 – 00:37:24]** use our post-training partners.

**[00:37:24 – 00:37:25]** These post-training partners that we put on the slide,

**[00:37:25 – 00:37:27]** we have vetted them.

**[00:37:27 – 00:37:30]** We call it the kitchen. I think we've got the inspectors in

**[00:37:30 – 00:37:34]** the room somewhere, Maran, if you want to talk to him about different

**[00:37:34 – 00:37:36]** types of post-training partners.

**[00:37:36 – 00:37:39]** We have the Agent Knowledge Foundation, how we integrate

**[00:37:39 – 00:37:42]** our libraries there, Routing and Optimization with Nemo

**[00:37:42 – 00:37:46]** Switchyard, and then again, Agent Safety and Security with OpenShell.

**[00:37:46 – 00:37:50]** And so we hope to grow that ecosystem, to get deeper on

**[00:37:50 – 00:37:53]** all kinds of use cases, because while coding was the first

**[00:37:53 – 00:37:57]** great killer app for generative AI, we're going to have to do the

**[00:37:57 – 00:37:58]** long tail of domains and use cases.

**[00:37:58 – 00:38:03]** And so we have to educate and empower everyone on how to create

**[00:38:03 – 00:38:05]** these super agents for themselves.

**[00:38:05 – 00:38:08]** Do you want to add anything?

**[00:38:08 – 00:38:11]** Okay, I think we have, like, three minutes left

**[00:38:11 – 00:38:16]** if there are any questions that anyone wanted to ask.

**[00:38:17 – 00:38:20]** I can't actually can't see, there's no questions.

**[00:38:20 – 00:38:25]** Questions? I think we have some.

**[00:38:25 – 00:38:29]** I have a question right here. I think it's hard to tell,

**[00:38:29 – 00:38:34]** people are taking pictures or asking a question.

**[00:38:37 – 00:38:41]** Help a friend, pass it down.

**[00:38:43 – 00:38:47]** I just wanted to ask, there's a lot of focuses on the Nemotron

**[00:38:47 – 00:38:50]** family of models.

**[00:38:51 – 00:38:56]** In the market of open source models, those models are still

**[00:38:56 – 00:38:59]** scoring somewhat below on intelligence, specifically,

**[00:38:59 – 00:39:02]** versus some of the frontier open source models.

**[00:39:02 – 00:39:10]** Is there... are we focusing here on training specific sort of Nemotron

**[00:39:10 – 00:39:12]** models to do specific things?

**[00:39:12 – 00:39:15]** Is that the focus versus getting the smartest type

**[00:39:15 – 00:39:16]** of Nemotron out there?

**[00:39:16 – 00:39:17]** Is that still a focus?

**[00:39:18 – 00:39:21]** Yeah, I mean, I think we want to aim for the best models possible.

**[00:39:21 – 00:39:23]** What we're doing, again, is this is a new type of software platform.

**[00:39:23 – 00:39:27]** And anything that we've done as a development platform, again,

**[00:39:27 – 00:39:30]** we open up the RL environments, everything that we use, the data

**[00:39:30 – 00:39:33]** sets, to the extent that we can.

**[00:39:33 – 00:39:36]** And what you'll notice with a lot of open weights models

**[00:39:36 – 00:39:38]** is they do not open source their RL environments.

**[00:39:38 – 00:39:40]** Or their data.

**[00:39:40 – 00:39:42]** Or their data. So you don't actually know

**[00:39:42 – 00:39:42]** what's in the weights.

**[00:39:42 – 00:39:45]** And so we are completely transparent about our data,

**[00:39:45 – 00:39:48]** where we got our data, how we created our data, and

**[00:39:48 – 00:39:53]** so we would love everyone, we have the Nemotron Coalition

**[00:39:53 – 00:39:55]** now that we're working with on Nemotron 4 to be able to improve

**[00:39:55 – 00:39:58]** the Nemotron models for Nemotron 4.

**[00:39:58 – 00:40:00]** But what we've heard from our post-training partners,

**[00:40:00 – 00:40:03]** and this is something that we've, the feedback we've

**[00:40:03 – 00:40:07]** gotten over and over again is that Nemotron is the most customizable.

**[00:40:07 – 00:40:09]** It is so easy to customize it, and I think it's also

**[00:40:09 – 00:40:12]** because we open up these data sets in our environments.

**[00:40:12 – 00:40:15]** And so if you've ever taken a model and you've quantized it down,

**[00:40:15 – 00:40:18]** you have to do the same blend to be able to retrain accuracy back into

**[00:40:18 – 00:40:21]** the model so you don't have massive forgetting in certain areas.

**[00:40:21 – 00:40:25]** And so that's actually something that's really key about being

**[00:40:25 – 00:40:27]** able to train for Nemotron.

**[00:40:27 – 00:40:30]** Not just customize it, but then run it efficiently on

**[00:40:30 – 00:40:33]** your hardware, on your compute.

**[00:40:34 – 00:40:36]** Thank you for the talk.

**[00:40:36 – 00:40:38]** I have a question about the hardware protection

**[00:40:38 – 00:40:39]** part of OpenShell.

**[00:40:40 – 00:40:42]** Is that only for the Vera platform, or does it extend

**[00:40:42 – 00:40:45]** to the current generation of GPUs?

**[00:40:45 – 00:40:48]** Yeah, so the architecture we're talking about is one

**[00:40:48 – 00:40:52]** where you've got a second system.

**[00:40:52 – 00:40:54]** Think of it in self-driving cars.

**[00:40:54 – 00:40:56]** You have a perception system, and then you've got in the

**[00:40:57 – 00:41:01]** car a functional safety island that looks at the health of

**[00:41:01 – 00:41:04]** the perception system, to ensure the car fails safely.

**[00:41:04 – 00:41:07]** So we're just saying that architecture of having a CPU,

**[00:41:07 – 00:41:12]** could be any CPU, x86 or ARM, and a secondary safety system to

**[00:41:12 – 00:41:15]** look at how the agent is thinking, gives you an architecture where

**[00:41:15 – 00:41:17]** you can ensure things fail safely.

**[00:41:18 – 00:41:22]** And we think no matter which CPU you use, having that secondary

**[00:41:22 – 00:41:25]** safety system is helpful in mission-critical environments

**[00:41:25 – 00:41:28]** where you're doing offensive red teaming, or you're looking

**[00:41:28 – 00:41:31]** at models pre-alignment.

**[00:41:31 – 00:41:33]** And so really, that's what we're trying to advocate for.

**[00:41:33 – 00:41:36]** Thank you, Carrie and Justin.

**[00:41:36 – 00:41:39]** Let's have a big round of applause for Carrie and Justin.

**[00:41:39 – 00:41:39]** Thank you.

