# How to Run Open Models and Agentic AI Efficiently at the Edge ## Abstract The edge is no longer just an inference endpoint—it's becoming an agent host. In this session, we'll highlight new software capabilities for building AI agents across the NVIDIA Jetson platform—from Orin to Thor. Our experts will show how NVIDIA NemoClaw on Jetson delivers a secure, enterprise-grade foundation for always-on physical AI agents without cloud dependency. You’ll learn practical techniques—from leveraging a lean Yocto-based OS to full-stack memory optimizations—designed to unlock more headroom for running larger models, more pipelines, and more capable agents on small edge devices. Together, these elements help developers accelerate physical AI application deployment while reducing total cost of ownership at scale. Whether you’re building humanoid robots, medical devices, or factory automation systems, you’ll gain concrete patterns to take agentic AI into production. For more GTC Taipei sessions like this, make sure to explore our Physical AI Days programming. ## AI Summary - NVIDIA Jetson was described as an edge AI computer designed for deployment, not for simulation or model training. - JetPack 7.2, released just before Computex and GTC Taipei, introduced support for NeMoClaw, a popular open-source project for agentic AI, and Jetson-specific skills to optimize memory usage. - The presentation highlighted the importance of memory optimization due to the high cost of DRAM, and how Jetson’s new features could help users choose more cost-efficient modules. ## Transcript **[00:00:03 – 00:00:08]** right hello everybody uh my name is Chitoku Yato. **[00:00:08 – 00:00:14]** I am a Jetson technical marketing manager uh here **[00:00:15 – 00:00:17]** NVIDIA, NVIDIA headquarter, actually. **[00:00:18 – 00:00:19]** I'm based in Santa Clara headquarter. **[00:00:19 – 00:00:25]** I'm leading this team of Jetson TME, helping all the developers and **[00:00:25 – 00:00:28]** customer who are working on Jetson. **[00:00:28 – 00:00:31]** Today I'm joined by Asir, my colleague. **[00:00:32 – 00:00:35]** He is ready to give a really exciting demo at the end of **[00:00:36 – 00:00:39]** this presentation. **[00:00:39 – 00:00:44]** So we have a couple of topics that I want to **[00:00:44 – 00:00:48]** cover, so let's get going. **[00:00:49 – 00:00:53]** In this course, you'll discover how NVIDIA's Jetson platform **[00:00:53 – 00:00:56]** is unlocking the full potential **[00:00:57 – 00:01:00]** Okay, that wasn't expected. **[00:01:01 – 00:01:05]** Yeah, if you can mute through the rest of the **[00:01:05 – 00:01:08]** slide, that would be great. **[00:01:08 – 00:01:14]** Okay, so I need to first address an elephant in the room. **[00:01:14 – 00:01:18]** How many of you know NVIDIA Jetson? **[00:01:18 – 00:01:20]** Oh, quite some people. **[00:01:20 – 00:01:22]** I'm glad. **[00:01:22 – 00:01:26]** How many people are already working on Jetson? **[00:01:26 – 00:01:30]** Great, great, yeah, that's what we'd like to see. **[00:01:31 – 00:01:36]** Yeah, so Jetson is NVIDIA's edge AI computer. **[00:01:36 – 00:01:40]** Jetson Fung always talks about three computer problems. **[00:01:40 – 00:01:44]** First and second computer work together to simulate **[00:01:44 – 00:01:47]** and train a model, a deep learning model, in the data **[00:01:47 – 00:01:50]** center or on your workstation. **[00:01:50 – 00:01:56]** The result is your trained or fine-tuned model. **[00:01:56 – 00:02:02]** And then some of them would go deployed in the physical world. **[00:02:02 – 00:02:05]** That's where NVIDIA Jetson comes in. **[00:02:05 – 00:02:08]** NVIDIA Jetson is a deployment platform. **[00:02:08 – 00:02:13]** That's where you would put your model to run or to **[00:02:13 – 00:02:15]** run the inference. **[00:02:15 – 00:02:25]** So that might be your AOI equipment working in the factory or real-time **[00:02:25 – 00:02:30]** surveyor running on drone. **[00:02:30 – 00:02:35]** So one of them actually looks like this. **[00:02:35 – 00:02:41]** It's a module that we ship to our customers and partners. **[00:02:41 – 00:02:45]** Many of them are here in Taiwan. **[00:02:45 – 00:02:50]** And those that you might have seen, and then we actually sell **[00:02:50 – 00:02:58]** downstairs, those are the developer kit for developer partners to get **[00:02:58 – 00:03:03]** started and develop their software. **[00:03:04 – 00:03:08]** And we often get asked, what is the difference between **[00:03:08 – 00:03:14]** Jetson and, well, Jetson Thor specifically, and DGX Spark? **[00:03:14 – 00:03:19]** Because Jetson Thor and DGX Spark both come with 128 gigabyte **[00:03:19 – 00:03:25]** memory, and they both have Blackwell generation GPU. **[00:03:25 – 00:03:32]** The way I'd like to characterize is that DGX Spark can cover **[00:03:32 – 00:03:37]** everything, all the way from the simulation that you would run **[00:03:37 – 00:03:42]** on your desktop to, well, not much on the deployment side, but like **[00:03:42 – 00:03:46]** training the model, fine-tuning the model, do all kinds of experiments **[00:03:46 – 00:03:50]** while you are on your desk. **[00:03:50 – 00:03:55]** Jetson, on the other hand, it is not to do a simulation. **[00:03:55 – 00:04:01]** It is not to run ISAAC simulator or anything, but it is really designed **[00:04:01 – 00:04:03]** to be the deployment platform. **[00:04:03 – 00:04:06]** So this is where all the inference would happen. **[00:04:06 – 00:04:11]** You may do the optimization on this, of course, but yes, **[00:04:11 – 00:04:16]** this is where, and then you will see, this is where this comes **[00:04:16 – 00:04:21]** in, this gets plugged into the final product of yours, and this is **[00:04:21 – 00:04:26]** all where the AI model would run. **[00:04:26 – 00:04:30]** Okay, so that's a little more characterization. **[00:04:30 – 00:04:32]** This is a little bit wordy. **[00:04:32 – 00:04:33]** You can take a photo. **[00:04:33 – 00:04:37]** You can find this on Jetson AI Lab as well, one of the **[00:04:37 – 00:04:41]** websites that we manage, that Jetson TMET manages, so you **[00:04:41 – 00:04:44]** can come take a look. **[00:04:45 – 00:04:51]** So now that gets out of the way, we can sort of start talking **[00:04:51 – 00:04:58]** about the main theme of this whole Computex or even GTC Taipei, right? **[00:04:58 – 00:05:01]** So that's agentic AI. **[00:05:02 – 00:05:06]** Jetson started talking about, started off the keynote **[00:05:06 – 00:05:09]** speech with agentic AI. **[00:05:09 – 00:05:15]** And we are already in this agentic AI revolution. **[00:05:16 – 00:05:20]** Again, how I would like to characterize is that in the past, **[00:05:20 – 00:05:25]** or yesterday, what we were building was a little bit more purpose-built **[00:05:25 – 00:05:31]** for a single function, and we need to, well, humans needed to develop **[00:05:31 – 00:05:33]** everything, program everything. **[00:05:33 – 00:05:39]** But what we are seeing emerging today is more agentic, agentic **[00:05:39 – 00:05:44]** AI driven system and frankly this is really powerful, I feel **[00:05:44 – 00:05:49]** the sort of like impact of it even by myself when we are developing **[00:05:50 – 00:05:54]** our demonstration software. **[00:05:54 – 00:05:59]** What's great is that you can now count on the agent running **[00:05:59 – 00:06:03]** either in a cloud or even on the device itself to make **[00:06:03 – 00:06:06]** the judgment at that time. **[00:06:06 – 00:06:12]** So, to put it simple, you don't need to program everything. **[00:06:12 – 00:06:16]** The agent would do the judgment and then make a necessary **[00:06:16 – 00:06:18]** call at the spot. **[00:06:18 – 00:06:23]** It wouldn't even need a UI for all the use cases. **[00:06:24 – 00:06:29]** You may be able to just have your end user talk to it. **[00:06:29 – 00:06:34]** So that's as powerful as this agentic AI approach. **[00:06:34 – 00:06:38]** And we launched the new software for Jetson. **[00:06:38 – 00:06:41]** It's called JetPack 7.2. **[00:06:41 – 00:06:46]** Only two days before, we wanted to make it come at the time **[00:06:46 – 00:06:50]** of Computex and GTC Type-A. **[00:06:50 – 00:06:56]** And we introduce many new features along with JetPack 7.2. **[00:06:56 – 00:06:59]** Some of them are inside of the JetPack. **[00:06:59 – 00:07:02]** Some of them come outside of the JetPack. **[00:07:02 – 00:07:09]** I'd like to go through each of them just to sort of, yeah, **[00:07:09 – 00:07:17]** explain the impact of agentic AI and how you can appreciate it. **[00:07:17 – 00:07:22]** So the way it goes is that I'm going to explain all those **[00:07:22 – 00:07:25]** with the slide on the stage. **[00:07:25 – 00:07:30]** That takes place for another 20 minutes or so. **[00:07:30 – 00:07:33]** And then I will explain. **[00:07:33 – 00:07:38]** You will feel the impact of the Agentic AI when it **[00:07:38 – 00:07:40]** comes to Asir's demo. **[00:07:40 – 00:07:43]** So yeah, please hold on to it. **[00:07:43 – 00:07:49]** You will probably enjoy his great demonstration. **[00:07:49 – 00:07:54]** Okay, so first, Jetson going agentic. **[00:07:54 – 00:07:55]** What does that mean? **[00:07:55 – 00:07:59]** Well, the first thing we did is to support NemoClaw. **[00:07:59 – 00:08:02]** NemoClaw is one of the **[00:08:02 – 00:08:05]** This is a quickly growing, very popular open source project, **[00:08:05 – 00:08:10]** and this is a great agentic harness and especially great **[00:08:10 – 00:08:12]** for personal assistant. **[00:08:12 – 00:08:16]** Yeah, it's a great starting point, so we wanted to make **[00:08:16 – 00:08:21]** it as frictionless as possible, so this one single command **[00:08:21 – 00:08:24]** now runs smoothly on Jetson. **[00:08:24 – 00:08:27]** That's a starting point. **[00:08:27 – 00:08:33]** On top of that, we are introducing a bunch of Jetson Agentic **[00:08:33 – 00:08:37]** Skills, so this is something not part of the JetPack, but **[00:08:37 – 00:08:39]** comes with JetPack 7.2. **[00:08:40 – 00:08:44]** We release those on GitHub. **[00:08:44 – 00:08:49]** If you go to NVIDIA AI IoT GitHub organization, you will **[00:08:49 – 00:08:55]** find those two repo, one for host side, so that's to be installed **[00:08:55 – 00:09:02]** on your PC if you are customizing the Jetson Linux BSP or the image. **[00:09:02 – 00:09:08]** And the other one is for device, that is to be installed on **[00:09:08 – 00:09:13]** the runtime Jetson devices if you are developing on Jetson **[00:09:13 – 00:09:15]** or using it on a runtime. **[00:09:15 – 00:09:18]** Those skills are going to be pretty useful. **[00:09:18 – 00:09:23]** I will explain how that plays out. **[00:09:23 – 00:09:29]** Yeah, today many people now code or develop software. **[00:09:29 – 00:09:31]** with Coding Agents, right? **[00:09:31 – 00:09:38]** So I wanna ask, how many of you are using Codex? **[00:09:38 – 00:09:41]** Codex people, all right. **[00:09:41 – 00:09:44]** Cloud Code. **[00:09:44 – 00:09:48]** All right, how about Cursor? **[00:09:48 – 00:09:53]** Oh, oh, yeah, Cursor, I did, and I switched to Cloud Code. **[00:09:53 – 00:09:57]** But, okay, but yeah, many of you are already using them, **[00:09:57 – 00:09:59]** right, yeah, they are great. **[00:09:59 – 00:10:05]** But they tend to stumble upon some of the Jetson unique **[00:10:05 – 00:10:09]** things, because Jetson is an ARM-based computer. **[00:10:09 – 00:10:15]** ARM is getting quite popular even on the server side, so it's **[00:10:15 – 00:10:17]** getting to be less of an issue. **[00:10:17 – 00:10:22]** But still, still, sometimes, those generic coding agents **[00:10:22 – 00:10:28]** would miss the point and start sort of putting on... **[00:10:28 – 00:10:36]** x86 container or trying to go build the whole software **[00:10:36 – 00:10:40]** because it didn't find an ARM binary or where to look **[00:10:40 – 00:10:44]** for the ARM binary and stuff. **[00:10:46 – 00:10:52]** Skills make those coding agents even smarter, right? **[00:10:52 – 00:10:55]** They are now expert even on Jetson. **[00:10:55 – 00:11:03]** They know how to use Jetson, the software specific to Jetson, or the **[00:11:03 – 00:11:09]** interfaces that's made available on Jetson, hardware interfaces. **[00:11:09 – 00:11:14]** And one of the way you can actually take advantage of that is you **[00:11:14 – 00:11:20]** can use those skill to even trying to cut down the memory usage. **[00:11:20 – 00:11:23]** And this is very impactful today, right? **[00:11:23 – 00:11:27]** The rise of the memory cost, DRAM are sky high, and DRAM **[00:11:27 – 00:11:29]** cost are sky high. **[00:11:29 – 00:11:30]** Everybody wants to... **[00:11:31 – 00:11:34]** Spend a little more time on seriously trying to **[00:11:34 – 00:11:35]** optimize the memory. **[00:11:35 – 00:11:39]** Now, agent can of course help, and they can do even better **[00:11:39 – 00:11:44]** job with the skills that we are providing, because we ourselves **[00:11:44 – 00:11:49]** are using those skills and getting quite good results with it. **[00:11:51 – 00:11:54]** Yeah, memory optimization is crucial today. **[00:11:54 – 00:11:59]** LLM and VLMs, those are memory-hungry, **[00:11:59 – 00:12:01]** memory-intensive models. **[00:12:01 – 00:12:03]** And then again, DRAM is expensive. **[00:12:03 – 00:12:07]** So there is a all good reason why you want to spend time **[00:12:07 – 00:12:10]** or effort on memory optimization. **[00:12:10 – 00:12:13]** Was a little hard yesterday. **[00:12:13 – 00:12:17]** But today, with Coding Agent and with our skills that we **[00:12:17 – 00:12:22]** are publishing, it's getting even easier and more. **[00:12:23 – 00:12:26]** There is a good reason why you want to spend a little more time on it. **[00:12:26 – 00:12:29]** Because you will get the return. **[00:12:30 – 00:12:34]** What I mean by the return is especially the cost of **[00:12:34 – 00:12:36]** ownership, right? **[00:12:36 – 00:12:41]** It's not just about cutting down the time of the development, **[00:12:41 – 00:12:43]** but it goes all the way. **[00:12:43 – 00:12:46]** Not just developing the software, prototyping, but **[00:12:46 – 00:12:49]** all the ops and tests. **[00:12:49 – 00:12:58]** And you can eventually make your cost-efficient, cost-aware **[00:12:58 – 00:13:04]** choice of picking our range of NVIDIA Jetson module. **[00:13:04 – 00:13:08]** We offer many Jetson. **[00:13:09 – 00:13:14]** Modules like this with varying size of memory, if you are **[00:13:14 – 00:13:19]** successful in memory optimizing, instead of using 16GB model, **[00:13:19 – 00:13:22]** you could use 8GB model. **[00:13:22 – 00:13:26]** You can save money by doing that, in that way. **[00:13:26 – 00:13:32]** We'd like to see our customers, you know, successful in doing that. **[00:13:32 – 00:13:40]** So, that's one of the reasons why we want to push this approach. **[00:13:40 – 00:13:47]** Okay, so the next is, I want to tackle two at the same time. **[00:13:47 – 00:13:51]** So they're pretty impactful. **[00:13:51 – 00:13:56]** So currently, NVIDIA provides Jetson SOA, top of the **[00:13:56 – 00:13:57]** lineup, flagship. **[00:13:57 – 00:13:59]** This is, **[00:13:59 – 00:14:04]** NVIDIA Blackwell generation GPU enabled Jetson. **[00:14:05 – 00:14:11]** And others are Jetson AGX Orin and Jetson Orin Nano. **[00:14:12 – 00:14:18]** Actually, in between we have Orin NX as well, which is this. **[00:14:18 – 00:14:22]** So they are different generation product. **[00:14:22 – 00:14:26]** Orin is based on Ampere generation GPU. **[00:14:26 – 00:14:31]** So in that sense, it is three, four even years old. **[00:14:31 – 00:14:35]** However, we continue to ship the hardware, we continue **[00:14:35 – 00:14:37]** to support the software. **[00:14:37 – 00:14:44]** That is one thing NVIDIA Jetson team try to do and do great, **[00:14:44 – 00:14:45]** long-term support. **[00:14:46 – 00:14:50]** And in fact, we promise to keep shipping this module **[00:14:50 – 00:14:56]** until at least 2032, so that our customer are sure to be able to **[00:14:56 – 00:14:59]** continue build their product on it. **[00:14:59 – 00:15:02]** Now then, what's important comes even more important **[00:15:02 – 00:15:04]** in the software support, right? **[00:15:04 – 00:15:08]** With JetPack 7.2, we are now bringing the latest software **[00:15:09 – 00:15:12]** to the four-year-old Jetson Orin. **[00:15:12 – 00:15:17]** And then what this specifically brings is the **[00:15:17 – 00:15:24]** Orins are now capable of running all the mainstream **[00:15:24 – 00:15:27]** ARM binaries and containers. **[00:15:27 – 00:15:32]** Some people colloquially say SBSA, 64 SBSA architecture. **[00:15:32 – 00:15:39]** Yes, because of the ARM-based server getting ever popular, **[00:15:39 – 00:15:44]** we now see upstream projects like, say, PyTorch, VLLM. **[00:15:44 – 00:15:49]** They are all producing now Nitri builds and the official build, **[00:15:49 – 00:15:53]** both for x86 and ARM architectures. **[00:15:53 – 00:16:00]** So it was almost there for Jetson to be able to just **[00:16:00 – 00:16:02]** take advantage of it. **[00:16:02 – 00:16:09]** And SOA actually can do right now, today, even before the 7.2 release. **[00:16:10 – 00:16:17]** However, Orin had continued to need its specific build **[00:16:17 – 00:16:25]** for Orin specifically because it had a little bit different GPU, **[00:16:25 – 00:16:27]** particularly driver architecture. **[00:16:27 – 00:16:32]** So the mainstream ARM binary didn't work on it. **[00:16:33 – 00:16:36]** With 7.2, those are now finally enabled. **[00:16:36 – 00:16:43]** So you can actually pull the latest of the VLM container **[00:16:44 – 00:16:46]** and let it run on Jetson Orin. **[00:16:46 – 00:16:51]** Some people say this is like Orin now gets a new life. **[00:16:51 – 00:16:59]** It's really great news for all the people who had been working on Orin **[00:16:59 – 00:17:06]** generation Jetson and developing their product based on Jetson Orin. **[00:17:06 – 00:17:09]** So that's JetPack 7.2 support. **[00:17:09 – 00:17:12]** Oh, I should have moved to the other slide. **[00:17:12 – 00:17:17]** So that's ARM64-SBSA support for Orin. **[00:17:17 – 00:17:23]** The other feature is multi-instant GPU, sometimes like abbreviated **[00:17:23 – 00:17:31]** to be MIG, it's a CUDA feature, this has been available on the **[00:17:31 – 00:17:36]** server side GPU, if you are running some kind of mission-critical **[00:17:36 – 00:17:38]** or timing-critical **[00:17:38 – 00:17:42]** So, if you are running multiple models on Jetson and trying **[00:17:42 – 00:17:46]** to make them work best. **[00:18:09 – 00:18:13]** So that's now available on JetPack 7.2 as well. **[00:18:15 – 00:18:20]** Okay, finally, it's getting to be one of the exciting topic, and I **[00:18:20 – 00:18:24]** see I can talk more about it later, but let me just quickly go through. **[00:18:24 – 00:18:26]** Yocto. **[00:18:26 – 00:18:30]** Actually, I want to also again ask you, Yocto, how many of you **[00:18:30 – 00:18:34]** are familiar with Yocto project? **[00:18:34 – 00:18:35]** Okay, great. **[00:18:35 – 00:18:41]** Actually, now that brings me another question. **[00:18:43 – 00:18:50]** How many of you would classify yourself as AI developers? **[00:18:50 – 00:18:53]** AI developers, all right, great, great, great. **[00:18:53 – 00:18:59]** How about, like, how about all the people who would classify **[00:18:59 – 00:19:02]** themselves as embedded engineer? **[00:19:02 – 00:19:05]** Embedded engineer, anybody from the embedded side? **[00:19:05 – 00:19:08]** Okay, great, I found my friend over there. **[00:19:08 – 00:19:09]** Great, great. **[00:19:09 – 00:19:13]** Yes, one of the interesting thing is, Jetson is a very interesting **[00:19:13 – 00:19:18]** product where the two different domain engineers meet, and they **[00:19:18 – 00:19:23]** need to collaborate together, and that was also the reason why skills **[00:19:23 – 00:19:31]** is very impactful, because they would allow us to bridge the gap. **[00:19:31 – 00:19:35]** I don't know much about the AI side if I am coming from **[00:19:35 – 00:19:43]** the embedded field, but skills can bridge the gap, and vice versa. **[00:19:43 – 00:19:50]** So that's another reason why skills for the coding agent is impactful, **[00:19:50 – 00:19:53]** especially on product like Jetson. **[00:19:53 – 00:19:55]** All right, Yocto Project. **[00:19:55 – 00:19:58]** These are popular. **[00:19:58 – 00:20:03]** Embedded projects are great for making their product, **[00:20:03 – 00:20:09]** embedded product, fine-tuned for the real deployment, as **[00:20:09 – 00:20:13]** a product, as an instrument. **[00:20:13 – 00:20:16]** One of the things it focuses on doing is to come up with **[00:20:16 – 00:20:21]** a very slimmed-down version of the software footprint. **[00:20:22 – 00:20:27]** Today, what NVIDIA ships Jetson with is, on the software side, **[00:20:28 – 00:20:33]** is JetPack because we try to showcase all the great things **[00:20:33 – 00:20:39]** that comes with it that you can get the benefit on the NVIDIA GPU. **[00:20:39 – 00:20:45]** It packs a lot of SDKs, not only just CUDA, DEV, but all **[00:20:45 – 00:20:49]** the library and SDK on top of that. **[00:20:49 – 00:20:53]** So the latest JetPack is like a 28-gigabyte. **[00:20:53 – 00:20:56]** Some of our customers are like, oh, no, no, no, that's too big. **[00:20:56 – 00:21:00]** We need to shrink it down if we are to ship it in our product. **[00:21:00 – 00:21:04]** Yes, that's certainly the way. **[00:21:05 – 00:21:09]** What comes in as a rescue is this Yocto project. **[00:21:09 – 00:21:15]** It allows you to make a very slimmed-down, purpose-built, **[00:21:15 – 00:21:19]** robust image for your particular application. **[00:21:19 – 00:21:24]** It gets shipped with only the library driver needed for you. **[00:21:24 – 00:21:27]** You are in control. **[00:21:27 – 00:21:35]** You can choose to not even have a read-write file system. **[00:21:35 – 00:21:40]** Or package manager, because those are for development. **[00:21:40 – 00:21:45]** So you can see how this has been. **[00:21:45 – 00:21:50]** One of the popular framework and feature requested by our **[00:21:50 – 00:21:55]** partners with their series about making a product based on Jetson. **[00:21:55 – 00:22:00]** It has been sort of community supported, but now NVIDIA **[00:22:00 – 00:22:03]** is working with the Yocto project and it is now officially **[00:22:03 – 00:22:06]** supported by NVIDIA. **[00:22:07 – 00:22:15]** Yeah, I'm pretty excited about this news and again the benefit **[00:22:15 – 00:22:21]** is quite something, you get to like really build, now you know **[00:22:21 – 00:22:26]** how your product can actually hit the market as you secure the method **[00:22:26 – 00:22:32]** to deploy your product in the world in a secure and replicable manner. **[00:22:33 – 00:22:38]** One of the great benefits also is the memory reduction, right? **[00:22:39 – 00:22:43]** Again, this is very important for **[00:22:43 – 00:22:49]** today's cost-aware environment. **[00:22:50 – 00:22:57]** All right, so this is sort of a decision guide or table. **[00:22:58 – 00:23:01]** Frankly, if you are just start developing on Jetson and then test **[00:23:01 – 00:23:10]** out what it has to offer, please stay with L4T or our original BSP. **[00:23:10 – 00:23:14]** You can just install everything on Jetson if you especially **[00:23:14 – 00:23:19]** have a large enough NVMe. **[00:23:19 – 00:23:23]** Once you are certain that you would ship your product **[00:23:23 – 00:23:28]** in a manner that, you know, where the storage size matters, **[00:23:28 – 00:23:32]** or you need to like seriously cut down the size of the memory, then **[00:23:32 – 00:23:34]** Yocto comes to be in the picture. **[00:23:34 – 00:23:39]** When you start thinking about the deployment, you will probably **[00:23:39 – 00:23:41]** start working on Yocto. **[00:23:41 – 00:23:47]** Knowing that such option is available lets you focus on **[00:23:47 – 00:23:51]** the initial deployment, initial development on the AI side. **[00:23:51 – 00:23:54]** That's how I think. **[00:23:54 – 00:23:59]** Okay, lastly, we have this **[00:23:59 – 00:24:02]** great news for those who cares. **[00:24:02 – 00:24:06]** We had this product called **[00:24:06 – 00:24:12]** So under the Jetson AGX Orin umbrella, we had 32 gigabyte **[00:24:12 – 00:24:15]** variant and 64 gigabyte variant. **[00:24:15 – 00:24:18]** And as the name suggests, it comes with the different **[00:24:18 – 00:24:19]** size of the memory. **[00:24:19 – 00:24:25]** 64 is actually big, considered to be huge when it first came **[00:24:25 – 00:24:28]** out, again, three, four years ago. **[00:24:29 – 00:24:34]** So we had the 32 gigabyte version as well. **[00:24:34 – 00:24:39]** And it came with reduced performance. **[00:24:39 – 00:24:44]** However, with this new software, we sort of unlocked some of **[00:24:44 – 00:24:46]** the frequency limitation. **[00:24:46 – 00:24:52]** We managed to push the limit on the 32 gigabyte version of the module. **[00:24:52 – 00:24:57]** So you are actually getting more performance on the same product. **[00:24:58 – 00:25:02]** NVIDIA is not charging more, but you are getting more performance **[00:25:02 – 00:25:04]** out of the same hardware. **[00:25:04 – 00:25:10]** This is kind of similar to what we did almost two years **[00:25:10 – 00:25:12]** ago with Jetson Orin Nano. **[00:25:12 – 00:25:19]** When we came out with Orin Nano Super, we did exactly the same. **[00:25:19 – 00:25:22]** The picture is the same here as well. **[00:25:22 – 00:25:26]** You are getting more performance, but with the same hardware. **[00:25:26 – 00:25:34]** And this is, again, very important today, as memory is a really **[00:25:34 – 00:25:41]** expensive part and a big portion of your BOM cost. **[00:25:41 – 00:25:45]** Although NVIDIA's module is actually great because it **[00:25:45 – 00:25:46]** comes with memory. **[00:25:46 – 00:25:52]** You don't need to worry about getting or procuring memory **[00:25:52 – 00:25:56]** under this market condition. **[00:25:56 – 00:26:00]** However, in general, the memory is getting expensive, **[00:26:00 – 00:26:05]** so there is a good reason why you want to optimise it. **[00:26:05 – 00:26:09]** So once you are successful in reducing the memory, or **[00:26:09 – 00:26:13]** the memory footprint, memory usage, you now have a choice **[00:26:13 – 00:26:17]** of moving from the expensive version of an MDBuff product **[00:26:17 – 00:26:21]** to a more cost-efficient option. **[00:26:22 – 00:26:28]** And we are happy to guide our customer doing that because **[00:26:28 – 00:26:33]** Frankly, it is even getting harder for NVIDIA to find **[00:26:33 – 00:26:35]** enough memory to ship our product. **[00:26:35 – 00:26:41]** So the more we get to help our customer move to a lower **[00:26:41 – 00:26:47]** memory version, we get to help more users, our customers. **[00:26:47 – 00:26:54]** So that's how serious NVIDIA is in helping our customer in **[00:26:54 – 00:26:59]** today's memory scenario, weather. **[00:27:01 – 00:27:06]** Yes, you are getting almost the same, well, some of the models **[00:27:06 – 00:27:13]** are different, but across all those models that we benchmark, we are **[00:27:13 – 00:27:19]** getting almost similar performance as the upper bound product, **[00:27:19 – 00:27:22]** Jetson, AGX, all in 64 gigabytes. **[00:27:22 – 00:27:28]** So, yes, you can rest assured moving to 32GB if you have been **[00:27:28 – 00:27:35]** running on 64GB version when it comes to the compute performance. **[00:27:38 – 00:27:44]** So that wraps up my side of the presentation. **[00:27:44 – 00:27:48]** You can find some of the topic, especially about the memory **[00:27:48 – 00:27:51]** optimization, if you are interested in that, please come check **[00:27:51 – 00:27:53]** out the NVIDIA tech blog. **[00:27:53 – 00:28:01]** You can probably just Google NVIDIA tech blog memory optimization and **[00:28:01 – 00:28:06]** our article would come at the top. **[00:28:06 – 00:28:12]** As a place, if you are just genuinely interested in Jetson **[00:28:12 – 00:28:17]** overall, another great avenue is Jetson AI Lab. **[00:28:18 – 00:28:21]** Jetson AI Lab, again, if you Google it, that comes at the top. **[00:28:21 – 00:28:27]** That's our team managers and always try to put the latest **[00:28:27 – 00:28:28]** information on it. **[00:28:28 – 00:28:31]** This is, frankly, the best source of information when **[00:28:31 – 00:28:37]** it comes to developing on Jetson, so please come check out as well. **[00:28:37 – 00:28:41]** So that wraps up my portion of the talk. **[00:28:41 – 00:28:46]** It's about five features that we brought together with JetPack 7.2 **[00:28:46 – 00:28:49]** and how that impacts our customers. **[00:28:49 – 00:28:55]** Now I'd like to switch our presentation to Asiris. **[00:28:55 – 00:29:00]** And he is ready to roll his exciting demo. **[00:29:00 – 00:29:02]** Take it away. **[00:29:20 – 00:29:22]** Welcome to NVIDIA GTC Taipei. **[00:29:22 – 00:29:24]** My name is Bruce. **[00:29:24 – 00:29:28]** I work on the sales team here at NVIDIA Taipei. **[00:29:28 – 00:29:33]** Today, I will be your host for this session. **[00:29:36 – 00:29:41]** Welcome to GTC Taipei, I'm NVIDIA Sales, I'm Bruce. **[00:29:41 – 00:29:46]** Now, let's welcome our host today, Chitoku-san, please. **[00:29:53 – 00:29:55]** Hello everybody. **[00:29:55 – 00:29:57]** My name is Chitoku Yato. **[00:29:57 – 00:30:02]** I am Jetson technical marketing manager here. **[00:30:04 – 00:30:06]** NVIDIA headquarter, actually. **[00:30:06 – 00:30:08]** I'm based in Santa Clara headquarter. **[00:30:08 – 00:30:12]** I'm leading this team of Jetson TME, helping all **[00:30:12 – 00:30:17]** the developers and customers who are working on Jetson. **[00:30:17 – 00:30:20]** Today I'm joined by Asir, my colleague. **[00:30:20 – 00:30:24]** He is ready to give a really exciting demo at the end of **[00:30:24 – 00:30:27]** this presentation. **[00:30:28 – 00:30:32]** So we have a couple of topics that I want to **[00:30:32 – 00:30:36]** cover, so let's get going. **[00:30:38 – 00:30:41]** In this course, you'll discover how NVIDIA's Jetson platform **[00:30:41 – 00:30:45]** is unlocking the full potential **[00:30:45 – 00:30:49]** OK, that wasn't expected. **[00:30:49 – 00:30:53]** Yeah, if you can mute through the rest of the **[00:30:53 – 00:30:56]** slide, that would be great. **[00:30:57 – 00:31:02]** OK, so I need to first address an elephant in the room. **[00:31:02 – 00:31:07]** How many of you know NVIDIA Jetson? **[00:31:07 – 00:31:08]** Oh, quite some people. **[00:31:08 – 00:31:10]** I'm glad. **[00:31:10 – 00:31:15]** How many people are already working on Jetson? **[00:31:15 – 00:31:16]** Great, great. **[00:31:16 – 00:31:19]** Yeah, that's what we'd like to see. **[00:31:19 – 00:31:25]** Yeah, so Jetson is NVIDIA's edge AI computer. **[00:31:25 – 00:31:29]** Jetson Fung always talks about three computer problems. **[00:31:29 – 00:31:33]** First and second computer work together to simulate and train a **[00:31:33 – 00:31:39]** model, a deep learning model in the data center or on your workstation. **[00:31:39 – 00:31:44]** The result is your trained or fine-tuned model. **[00:31:44 – 00:31:50]** And then some of them would go deployed in the physical world. **[00:31:50 – 00:31:53]** That's where NVIDIA Jetson comes in. **[00:31:53 – 00:31:56]** NVIDIA Jetson is a deployment platform. **[00:31:56 – 00:32:01]** That's where you would put your model to run or to **[00:32:01 – 00:32:04]** run the inference. **[00:32:04 – 00:32:10]** So that might be your A or I equipment working in the **[00:32:10 – 00:32:19]** factory or real-time surveyor running on drone. **[00:32:19 – 00:32:24]** So one of them actually looks like this. **[00:32:24 – 00:32:30]** It's a module that we ship to our customers and partners. **[00:32:30 – 00:32:34]** Many of them are here in Taiwan. **[00:32:34 – 00:32:39]** And those that you might have seen, and then we actually sell **[00:32:39 – 00:32:47]** downstairs, those are the developer kit for developer partners to get **[00:32:47 – 00:32:51]** started and develop their software. **[00:32:52 – 00:32:56]** And we often get asked, what is the difference between **[00:32:56 – 00:33:02]** Jetson and, well, Jetson Thor specifically, and DGX Spark? **[00:33:03 – 00:33:08]** Because Jetson Thor and DGX Spark both come with 128-gigabyte **[00:33:08 – 00:33:14]** memory, and they both have Blackwell generation GPU. **[00:33:14 – 00:33:20]** The way I'd like to characterize is that DGX Spark can cover **[00:33:20 – 00:33:25]** everything, all the way from the simulation that you would run **[00:33:25 – 00:33:30]** on your desktop to, well, not much on the deployment side, but like **[00:33:30 – 00:33:35]** training the model, fine-tuning the model, do all kinds of experiments **[00:33:35 – 00:33:38]** while you are on your desk. **[00:33:39 – 00:33:43]** Jetson, on the other hand, it is not to do a simulation. **[00:33:43 – 00:33:49]** It is not to run ISAAC simulator or anything, but it is really designed **[00:33:49 – 00:33:51]** to be the deployment platform. **[00:33:51 – 00:33:55]** So this is where all the inference would happen. **[00:33:55 – 00:34:00]** You may do the optimization on this, of course, but yes, **[00:34:00 – 00:34:04]** this is where, and then you will see, this is where this comes **[00:34:04 – 00:34:10]** in, this gets plugged into the final product of yours, and this is **[00:34:10 – 00:34:14]** all where the AI model would run. **[00:34:14 – 00:34:19]** Okay, so that's a little more characterization. **[00:34:19 – 00:34:20]** This is a little bit wordy. **[00:34:20 – 00:34:22]** You can take a photo. **[00:34:22 – 00:34:26]** You can find this on Jetson AI Lab as well. **[00:34:26 – 00:34:29]** One of the websites that we manage, that Jetson TMET manages, **[00:34:29 – 00:34:33]** so you can come take a look. **[00:34:34 – 00:34:39]** So now that gets out of the way, we can sort of start talking **[00:34:39 – 00:34:47]** about the main theme of this whole Computex or even GTC Taipei, right? **[00:34:47 – 00:34:50]** So that's agentic AI. **[00:34:50 – 00:34:54]** Jetson started talking about, started off the keynote **[00:34:55 – 00:34:57]** speech with agentic AI. **[00:34:57 – 00:35:04]** And we are already in this agentic AI revolution. **[00:35:04 – 00:35:08]** Again, how I would like to characterize is that in the past, **[00:35:08 – 00:35:14]** or yesterday, what we were building was a little bit more purpose-built **[00:35:14 – 00:35:19]** for a single function, and we need to, well, humans needed to develop **[00:35:19 – 00:35:21]** everything, program everything. **[00:35:22 – 00:35:28]** But what we are seeing emerging today is more agentic, agentic **[00:35:28 – 00:35:30]** AI-driven system. **[00:35:30 – 00:35:32]** And frankly, this is really powerful. **[00:35:32 – 00:35:37]** I feel the sort of like impact of it even by myself **[00:35:37 – 00:35:42]** when we are developing our demonstration software. **[00:35:43 – 00:35:47]** What's great is that you can now count on the agent running **[00:35:47 – 00:35:52]** either in a cloud or even on the device itself to make **[00:35:52 – 00:35:54]** the judgment at that time. **[00:35:54 – 00:36:00]** So to put it simple, you don't need to program everything. **[00:36:00 – 00:36:05]** The agent would do the judgment and then make a necessary **[00:36:05 – 00:36:07]** call at the spot. **[00:36:07 – 00:36:12]** We don't even need a UI for all the use cases. **[00:36:12 – 00:36:18]** You may be able to just have your end user talk to it. **[00:36:18 – 00:36:22]** So that's as powerful as this agentic AI approach. **[00:36:22 – 00:36:27]** And we launched the new software for Jetson. **[00:36:27 – 00:36:29]** It's called JetPack 7.2. **[00:36:29 – 00:36:35]** Only two days before, we wanted to make it come at the time **[00:36:35 – 00:36:38]** of Computex and GTC Taipei. **[00:36:38 – 00:36:44]** And we introduce many new features along with JetPack 7.2. **[00:36:44 – 00:36:47]** Some of them are inside of the JetPack. **[00:36:47 – 00:36:51]** Some of them come outside of the JetPack. **[00:36:51 – 00:36:57]** I'd like to go through each of them just to sort of, yeah, **[00:36:57 – 00:37:05]** explain the impact of agentic AI and how you can appreciate it. **[00:37:06 – 00:37:10]** So the way it goes is that I'm going to explain all those **[00:37:10 – 00:37:14]** with the slide on the stage. **[00:37:14 – 00:37:18]** That takes place for like another 20 minutes or so. **[00:37:19 – 00:37:22]** And then, I will explain. **[00:37:22 – 00:37:26]** You will feel the impact of the Asian TK AI when **[00:37:26 – 00:37:28]** it comes to Asir's demo. **[00:37:28 – 00:37:32]** So, yeah, please hold on to it. **[00:37:32 – 00:37:38]** You will probably enjoy his great demonstration. **[00:37:38 – 00:37:42]** Okay, so first, Jetson going agentic. **[00:37:42 – 00:37:44]** What does that mean? **[00:37:44 – 00:37:48]** Well, the first thing we did is to support NemoClaw. **[00:37:48 – 00:37:50]** NemoClaw is one of the **[00:37:50 – 00:37:55]** Quickly growing, very popular open source project, and this is a great **[00:37:55 – 00:38:01]** agentic harness, and especially great for personal assistant. **[00:38:01 – 00:38:05]** Yeah, it's a great starting point, so we wanted to make **[00:38:05 – 00:38:09]** it as frictionless as possible, so this one single command **[00:38:09 – 00:38:13]** now runs smoothly on Jetson. **[00:38:13 – 00:38:16]** That's a starting point. **[00:38:16 – 00:38:21]** On top of that, we are introducing a bunch of Jetson Agentic **[00:38:21 – 00:38:26]** Skills, so this is something not part of the JetPack, but **[00:38:26 – 00:38:28]** comes with JetPack 7.2. **[00:38:28 – 00:38:32]** We release those on GitHub. **[00:38:33 – 00:38:37]** If you go to NVIDIA AI IoT GitHub organization, you will **[00:38:38 – 00:38:44]** find those two repos, one for host side, so that's to be installed **[00:38:44 – 00:38:51]** on your PC if you are customizing the Jetson Linux BSP or the image. **[00:38:52 – 00:38:56]** And the other one is for device that is to be installed on **[00:38:56 – 00:38:58]** the runtime Jetson devices. **[00:38:58 – 00:39:05]** If you are developing on Jetson or using it on a runtime, those skills **[00:39:05 – 00:39:06]** are going to be pretty useful. **[00:39:06 – 00:39:12]** I will explain how that plays out. **[00:39:12 – 00:39:18]** Yeah, today, many people now code or develop software with **[00:39:18 – 00:39:21]** coding agents, right? **[00:39:21 – 00:39:27]** Actually, I want to ask, how many of you are using Codex? **[00:39:27 – 00:39:29]** Codex people, all right. **[00:39:29 – 00:39:32]** Cloud Code. **[00:39:33 – 00:39:36]** How about Cursor? **[00:39:36 – 00:39:39]** Oh, you have Cursor. **[00:39:39 – 00:39:42]** I did, and I switched to Cloud Code. **[00:39:42 – 00:39:46]** But, okay, but yeah, many of you are already using them, right? **[00:39:47 – 00:39:48]** Yeah, they are great. **[00:39:48 – 00:39:53]** But they tend to stumble upon some of the Jetson unique **[00:39:53 – 00:39:57]** things, because Jetson is an ARM-based computer. **[00:39:57 – 00:40:04]** ARM is getting quite popular even on the server side, so it's **[00:40:04 – 00:40:06]** getting to be less of an issue. **[00:40:06 – 00:40:10]** But still, still, sometimes, those generic coding agents **[00:40:10 – 00:40:16]** would miss the point and start sort of putting on... **[00:40:16 – 00:40:23]** x86 container or you know trying to go build the whole **[00:40:23 – 00:40:28]** software because it didn't find an ARM binary or where to **[00:40:28 – 00:40:32]** look for the ARM binary and stuff. **[00:40:35 – 00:40:40]** Skills make those coding agents even smarter, right? **[00:40:40 – 00:40:43]** They are now expert even on Jetson. **[00:40:43 – 00:40:52]** They know how to use Jetson, the software specific to Jetson, or the **[00:40:52 – 00:40:58]** interfaces that's made available on Jetson, hardware interfaces. **[00:40:58 – 00:41:02]** And one other way you can actually take advantage of that **[00:41:02 – 00:41:09]** is you can use those skills to even try to cut down the memory usage. **[00:41:09 – 00:41:12]** And this is very impactful today, right? **[00:41:12 – 00:41:16]** The rise of the memory costs, DRAM are sky high, and DRAM **[00:41:16 – 00:41:17]** costs are sky high. **[00:41:18 – 00:41:22]** Everybody wants to spend a little more time on seriously **[00:41:22 – 00:41:23]** trying to optimize the memory. **[00:41:23 – 00:41:26]** Now, agent can, of course, help. **[00:41:26 – 00:41:30]** And they can do an even better job with the skills that we **[00:41:30 – 00:41:34]** are providing because we ourselves are using those skills and **[00:41:34 – 00:41:38]** getting quite good results with it. **[00:41:40 – 00:41:42]** Yeah, memory optimization is crucial today. **[00:41:42 – 00:41:47]** LLM and VLMs, those are memory-hungry, **[00:41:47 – 00:41:50]** memory-intensive models. **[00:41:50 – 00:41:52]** And then again, DRAM is expensive. **[00:41:52 – 00:41:56]** So there is a all good reason why you want to spend time **[00:41:56 – 00:41:58]** or effort on memory optimization. **[00:41:58 – 00:42:01]** Was a little hard yesterday. **[00:42:01 – 00:42:05]** But today, with Coding Agent and with our skills that we **[00:42:05 – 00:42:11]** are publishing, it's getting even easier and more. **[00:42:11 – 00:42:15]** There is a good reason why you want to spend a little more time on it. **[00:42:15 – 00:42:18]** Because you will get the return. **[00:42:19 – 00:42:23]** What I mean by the return is especially the cost of **[00:42:23 – 00:42:24]** ownership, right? **[00:42:24 – 00:42:30]** It's not just about cutting down the time of the development, **[00:42:30 – 00:42:32]** but it goes all the way. **[00:42:32 – 00:42:35]** Not just developing the software, prototyping, but **[00:42:35 – 00:42:38]** all the ops and tests. **[00:42:38 – 00:42:46]** And you can eventually make your cost-efficient, cost-aware **[00:42:46 – 00:42:52]** choice of picking our range of NVIDIA Jetson module. **[00:42:52 – 00:42:57]** We offer many Jetson **[00:42:57 – 00:43:02]** Modules like this with varying size of memory, if you are **[00:43:02 – 00:43:08]** successful in memory optimising, instead of using 16 gigabyte model, **[00:43:08 – 00:43:10]** you could use 8 gigabyte model. **[00:43:10 – 00:43:15]** You can save money by doing that in that way. **[00:43:15 – 00:43:20]** We'd like to see our customers are successful in doing that. **[00:43:20 – 00:43:25]** So that's one of the reasons why we want to push this.