# Disney’s Olaf: From the Screen to Reality via Physical AI ## Abstract The translation of animated characters like Olaf, or the design of characters like our BDX Droids, requires balancing artistic fidelity with the constraints of the physical world. This talk explores how these robotic characters achieve that balance through the integration of modular mechatronics and deep reinforcement learning. Leveraging GPU-accelerated simulation to handle complex mechanisms, we train control policies that enable our characters to self-balance and self-regulate temperature. We further demonstrate progress in believable, autonomous navigation and human-robot interaction driven by onboard perception. ## AI Summary - The Walt Disney Company presented on the technology behind robotic characters, specifically OLAF and the BDX druids, highlighting the use of modular hardware components and reinforcement learning to quickly build and train these robots. - The team demonstrated how reinforcement learning connected mechatronics design and animation, enabling the robots to perform stylized movements and interact with guests in a believable manner. - For OLAF, the team introduced thermal dynamics modeling to prevent overheating and impact reduction rewards to minimize noise, ensuring the character stayed true to its animated counterpart. - The presentation also covered the development of the Camino simulator, which combined the simulation of mechanical and thermal dynamics, allowing for more robust and efficient training of robotic characters. - The team discussed the potential of using large language models and multimodal reasoning to enhance the autonomy and interaction capabilities of robotic characters, though this was not yet implemented for OLAF. - The speaker emphasized the importance of the team's expertise and collaboration, including both the technical and creative aspects, in bringing these robotic characters to life in Disney parks. ## Transcript **[00:00:11 – 00:00:14]** Good morning. **[00:00:15 – 00:00:17]** Yeah, thanks so much, Akhil. **[00:00:17 – 00:00:20]** It's the third year in a row that we get to present here, **[00:00:20 – 00:00:24]** and it's very humbling to be able to do that. **[00:00:24 – 00:00:27]** And we have a really amazing partnership with NVIDIA, so **[00:00:27 – 00:00:30]** I'll highlight some of the work that we actually do together. **[00:00:30 – 00:00:35]** So I'm going to talk about OLAF, but I'm also going to talk about **[00:00:35 – 00:00:36]** robotic characters in general. **[00:00:37 – 00:00:39]** So it's important that our technology that we build is **[00:00:39 – 00:00:41]** basically... Oh, could we go back? **[00:00:41 – 00:00:45]** Alright, it's advancing. **[00:00:45 – 00:00:49]** One more back. **[00:00:50 – 00:00:53]** So you're going to talk about technology behind robotic **[00:00:53 – 00:00:56]** characters, technology that helps, that is agnostic to **[00:00:57 – 00:01:00]** the character that we build and helps to build them quickly. **[00:01:00 – 00:01:04]** So the theme here is to go from screen to reality, and **[00:01:04 – 00:01:08]** we use physical AI, and physical AI means actually something different **[00:01:08 – 00:01:12]** for the Walt Disney Company, so I'll explain what that means. **[00:01:12 – 00:01:16]** All right, so a robotic character, what is a robotic character? **[00:01:16 – 00:01:20]** So some of you might have seen a robotic character before, **[00:01:20 – 00:01:23]** definitely yesterday some of you have seen, or most **[00:01:23 – 00:01:25]** of you have seen OLAF. **[00:01:25 – 00:01:28]** But so let me briefly explain what a robotic character is and **[00:01:28 – 00:01:31]** let me use our BDX druids to do so. **[00:01:31 – 00:01:38]** So BDX druids are new to us because they can freely roam in our parks. **[00:01:38 – 00:01:40]** They are bipedal systems, they have actually five degrees **[00:01:40 – 00:01:42]** of freedom in each leg. **[00:01:42 – 00:01:46]** So they have one degree of freedom less than we humans do. **[00:01:46 – 00:01:49]** So walking is more challenging for them. **[00:01:49 – 00:01:52]** But what's unique about them is that they can walk **[00:01:52 – 00:01:56]** around, interact with objects, interact with our guests. **[00:01:56 – 00:02:01]** And you as a guest can co-direct a story, so you can influence a **[00:02:01 – 00:02:06]** story, and every time you meet such a druid, the story is different. **[00:02:06 – 00:02:12]** So they enable us at The Walt Disney Company to augment our lens. **[00:02:12 – 00:02:15]** All right, so some of you might know that already, but **[00:02:15 – 00:02:19]** the inspiration for the BDX droids was the BD-1. **[00:02:20 – 00:02:25]** So this character took actually first steps in a game, not **[00:02:25 – 00:02:27]** in a movie, so in a game. **[00:02:27 – 00:02:30]** It's an explorer droid that assists basically if you're **[00:02:30 – 00:02:35]** on a mission to explore the area, and it's also a companion. **[00:02:35 – 00:02:37]** And so that served as an inspiration for us **[00:02:37 – 00:02:39]** to do the BDX Druid. **[00:02:39 – 00:02:42]** You see clear differences here, but so the game character **[00:02:42 – 00:02:44]** is basically smaller. **[00:02:44 – 00:02:47]** We had some constraints, actuator sizes and so on, and thought **[00:02:47 – 00:02:53]** we could just build an adult version of a BD type Druid. **[00:02:53 – 00:02:58]** If you think of a puppy, you know, the BDX would be the adult version. **[00:02:58 – 00:03:02]** But so since we last talked, since we were here last year, **[00:03:02 – 00:03:06]** we brought actually the VDX droids for limited engagements **[00:03:06 – 00:03:08]** to all of our global parks. **[00:03:08 – 00:03:11]** So this is really exciting for my team and all of us **[00:03:11 – 00:03:15]** at Imagineering to bring these droids to all of our parks. **[00:03:16 – 00:03:19]** And even we brought them to a cruise ship, the first **[00:03:19 – 00:03:21]** cruise ship, the Vish. **[00:03:21 – 00:03:24]** The good thing is they're going to come back every now and **[00:03:24 – 00:03:27]** then to all of these global parks. **[00:03:27 – 00:03:29]** This is thrilling to us. **[00:03:29 – 00:03:33]** All right. But now a bit more about the technology. **[00:03:33 – 00:03:36]** So we built this technology or tools that help us to build **[00:03:36 – 00:03:40]** these robotic characters, our new robotic characters quickly. **[00:03:40 – 00:03:43]** So we want to be able to build custom characters quickly. **[00:03:43 – 00:03:45]** So that's a unique challenge. **[00:03:45 – 00:03:48]** And we want to do this within months, not years. **[00:03:48 – 00:03:51]** So let me explain how that works. **[00:03:51 – 00:03:54]** So at the core of this is this triangle. **[00:03:54 – 00:03:57]** So you have mechatronics design, so we use CAD software to **[00:03:57 – 00:03:59]** design our robots. **[00:03:59 – 00:04:03]** And then we can extract simulation representations from them. **[00:04:03 – 00:04:06]** And then what's unique about our use case is that we have **[00:04:06 – 00:04:09]** always artistic interfaces. **[00:04:09 – 00:04:14]** So we use animation, basically motion targets that we want **[00:04:14 – 00:04:16]** these robots to be able to do. **[00:04:16 – 00:04:19]** And then, what connects these two disciplines is **[00:04:19 – 00:04:21]** reinforcement learning. **[00:04:21 – 00:04:24]** So reinforcement learning is really bringing this together **[00:04:24 – 00:04:25]** and allows us to iterate quickly. **[00:04:25 – 00:04:29]** So let me briefly explain that and use the BDX druids for that. **[00:04:29 – 00:04:34]** So to build custom robots quickly, we use modular hardware components. **[00:04:34 – 00:04:37]** So we have actuators or a family of actuators, and we also have **[00:04:38 – 00:04:40]** sensors and a family of sensors. **[00:04:40 – 00:04:44]** And we understand these actuators and sensors well in our software **[00:04:44 – 00:04:48]** stack, meaning in the runtime stack, but also in the offline **[00:04:48 – 00:04:51]** tools that we use, for example, for reinforcement learning. **[00:04:51 – 00:04:52]** That's crucial. **[00:04:52 – 00:04:55]** You need to understand your hardware well. **[00:04:55 – 00:04:58]** And then you also have the brains and power on board, so we have a **[00:04:58 – 00:05:02]** Jetson on board, that's incredibly important for autonomy, so that's **[00:05:02 – 00:05:06]** a connection to NVIDIA, we actually get help from NVIDIA also, when **[00:05:06 – 00:05:11]** it comes to runtime systems and so on, and we have a battery on board **[00:05:11 – 00:05:13]** that lasts for about two hours. **[00:05:13 – 00:05:16]** Before you have to recharge these components are agnostic **[00:05:16 – 00:05:20]** to the character but so one thing that is specific to the **[00:05:20 – 00:05:23]** character are the so-called show functions so we don't want to just **[00:05:23 – 00:05:27]** build functional robots we want to build robots that perform on **[00:05:27 – 00:05:31]** stage so we have speakers on board a headlamp for scanning we have **[00:05:31 – 00:05:35]** eyes that we can animate and we have antennas that we can actuate **[00:05:35 – 00:05:39]** And so speakers are quite important because you want to hear **[00:05:39 – 00:05:42]** a druid and not a mechanical robot. **[00:05:42 – 00:05:45]** But those are specific to the character, rest is agnostic **[00:05:45 – 00:05:46]** to the character. **[00:05:47 – 00:05:50]** So animation is an important key ingredient. **[00:05:50 – 00:05:54]** We create technology that always has a creative interface. **[00:05:54 – 00:05:58]** We want to make sure that our amazing world talent when it comes **[00:05:58 – 00:06:03]** to animation can create content for these robotic characters. **[00:06:03 – 00:06:05]** So animation is key. **[00:06:05 – 00:06:09]** And so the goal for our robots is basically to be able to **[00:06:09 – 00:06:13]** control them, remote control them, similar to how you control a game **[00:06:13 – 00:06:14]** character, a virtual character. **[00:06:14 – 00:06:19]** So we can use procedural animation to create the target motions **[00:06:19 – 00:06:22]** that we want these robots to be able to do. **[00:06:22 – 00:06:24]** And so you want to be able to control, for example, **[00:06:24 – 00:06:27]** the VDX Droids with two joystick controllers. **[00:06:27 – 00:06:30]** So with one you control the direction of walking and with **[00:06:30 – 00:06:35]** the other one the relative motion of the head relative to the body. **[00:06:35 – 00:06:38]** But this, if you would run this in a robot, the robot **[00:06:38 – 00:06:41]** would actually fall over, because this is kinematic input, it's not **[00:06:41 – 00:06:46]** physics informed, and that's where reinforcement learning comes in. **[00:06:46 – 00:06:49]** So with reinforcement learning you can connect these two worlds, you **[00:06:49 – 00:06:53]** get the simulation representation from the mechatronics design **[00:06:53 – 00:06:57]** and the animation input and then you can basically train **[00:06:57 – 00:06:59]** so-called control policies. **[00:06:59 – 00:07:01]** And that's actually quite a simple process, so you bring **[00:07:01 – 00:07:07]** your druid or your robot into a simulation environment. **[00:07:07 – 00:07:08]** And then you have two inputs here. **[00:07:08 – 00:07:12]** You want to be able to control the Droids, that's where the **[00:07:12 – 00:07:13]** operator commands come in. **[00:07:13 – 00:07:16]** They are inputs to a simple neural network, **[00:07:16 – 00:07:19]** a so-called control policy. **[00:07:19 – 00:07:22]** You also have the robot state as an input, and the policy **[00:07:23 – 00:07:26]** is a simple functional mapping that then goes from inputs **[00:07:26 – 00:07:31]** to outputs, and outputs in this context are actuator commands. **[00:07:31 – 00:07:35]** And so, we use reinforcement learning to train those, but **[00:07:35 – 00:07:38]** what's unique about our use case is that we send the same operator **[00:07:38 – 00:07:43]** commands during training also to an animation engine to define a **[00:07:43 – 00:07:48]** target state for every robot state, and then we can define rewards. **[00:07:48 – 00:07:52]** If you can follow an animation well, then you get high rewards, **[00:07:52 – 00:07:55]** especially also if the robot doesn't fall over. **[00:07:55 – 00:07:58]** But if the robot falls over, or if you don't follow these **[00:07:58 – 00:08:01]** stylized motions well, you get a lower reward. **[00:08:01 – 00:08:04]** And then reinforcement learning simply updates this mapping **[00:08:04 – 00:08:07]** by updating the weights. **[00:08:07 – 00:08:11]** And the important thing is that we randomize these commands. **[00:08:11 – 00:08:15]** So if you don't do that, there could be configurations or **[00:08:15 – 00:08:18]** combinations of the two joysticks or buttons and so on where **[00:08:18 – 00:08:20]** the robot could fail. **[00:08:20 – 00:08:23]** So you want to sample that entire space during training **[00:08:23 – 00:08:26]** so that you as an operator can do arbitrary combinations **[00:08:26 – 00:08:27]** of the operator commands. **[00:08:27 – 00:08:31]** While the robot is always stably balancing. **[00:08:31 – 00:08:34]** So here is the process, so we initialize these weights **[00:08:34 – 00:08:36]** basically with random numbers. **[00:08:36 – 00:08:40]** At the beginning the robot falls over, but over time **[00:08:40 – 00:08:44]** if you continue to update these weights, it not only balances **[00:08:44 – 00:08:49]** better, but slowly but surely can do the stylized reference motions **[00:08:49 – 00:08:52]** that our artists are interested in. **[00:08:52 – 00:08:56]** And so it's incredible how well this works and scales also. **[00:08:56 – 00:08:59]** That's again where GPUs come in. **[00:08:59 – 00:09:03]** So we use NVIDIA hardware for training here. **[00:09:03 – 00:09:07]** You can train so-called policies, these policies, you know, **[00:09:07 – 00:09:09]** in hours to a couple of days. **[00:09:09 – 00:09:11]** And we vary here the terrain. **[00:09:11 – 00:09:14]** We randomize the terrain so that they can stably balance even **[00:09:14 – 00:09:16]** if you don't have flagged ground. **[00:09:16 – 00:09:17]** And we also push them. **[00:09:17 – 00:09:21]** So we apply forces and torques that you see here in green and blue. **[00:09:21 – 00:09:24]** So that they know how to recover, do recovery steps **[00:09:24 – 00:09:25]** and don't fall over. **[00:09:25 – 00:09:28]** That's where the robustness comes from. **[00:09:28 – 00:09:31]** And so what amounts to a decade of training you can do in a **[00:09:31 – 00:09:34]** few hours, today's on-desktop GPUs. **[00:09:34 – 00:09:37]** Amazing, I think, what that unlocks. **[00:09:37 – 00:09:41]** Okay, so that gives you a really nice abstraction layer, so here... **[00:09:41 – 00:09:45]** You know, independent of what an operator does, even if you have **[00:09:45 – 00:09:49]** a very challenging environment, this robot learned to balance **[00:09:49 – 00:09:54]** through this process by randomizing the simulation, basically. **[00:09:54 – 00:09:56]** All right, so that opens up the question, well, you have **[00:09:56 – 00:09:58]** now an abstraction layer. **[00:09:58 – 00:10:00]** How can you make these characters more autonomous? **[00:10:00 – 00:10:03]** And I want to give you an update on our autonomous or **[00:10:03 – 00:10:05]** autonomous solutions there. **[00:10:06 – 00:10:09]** And so for us, it's not just about functional autonomy, **[00:10:09 – 00:10:11]** it's about believable autonomy. **[00:10:11 – 00:10:14]** You want to believe that this is a character on stage, not just **[00:10:15 – 00:10:18]** functionally going from A to B. **[00:10:18 – 00:10:22]** So one item that you can think about in an autonomy stack is **[00:10:22 – 00:10:27]** navigation and so I just explained that this you know abstraction **[00:10:27 – 00:10:31]** layer works quite well where you control remotely a droid with **[00:10:31 – 00:10:36]** operator commands so it abstracts basically the lower level controls **[00:10:36 – 00:10:40]** from a user but now the question is could you actually replace **[00:10:40 – 00:10:43]** this with a navigation policy? **[00:10:43 – 00:10:46]** And that takes from a brain navigation commands, for **[00:10:46 – 00:10:49]** example, waypoints along a trajectory as an input. **[00:10:49 – 00:10:52]** And then you want to be able to follow this trajectory, **[00:10:52 – 00:10:55]** while obviously you don't want to bump into anything **[00:10:55 – 00:10:57]** and you want to be in character. **[00:10:57 – 00:11:00]** And so to be able to do that, you not only need the robot **[00:11:00 – 00:11:03]** state, you also need the state of the environment. **[00:11:03 – 00:11:05]** You need perception models. **[00:11:05 – 00:11:07]** So you need a sensor model. **[00:11:07 – 00:11:12]** So a sensor model then takes the simulated environment **[00:11:12 – 00:11:16]** as an input and provides a sensor state that then serves as **[00:11:16 – 00:11:20]** an input in its navigation policy. **[00:11:20 – 00:11:23]** So then we apply actually something very similar to what **[00:11:23 – 00:11:26]** we do if we train a control policy. **[00:11:26 – 00:11:30]** We send the same navigation commands to an artistic goal **[00:11:30 – 00:11:33]** engine that provides us then with a target state. **[00:11:33 – 00:11:36]** And this target state could be discrete, that discrete **[00:11:36 – 00:11:40]** time steps you want to be at a certain location. **[00:11:40 – 00:11:43]** That's sort of a simple case, and could be a partial state **[00:11:43 – 00:11:45]** of the robot, for example, just the pelvis, or it could **[00:11:45 – 00:11:49]** be the full state, and it could be dense inputs, because we want **[00:11:49 – 00:11:53]** to have a robot not just go from A to B, but do this in style, with **[00:11:53 – 00:11:57]** a personality, and maybe that's not the straight path, actually. **[00:11:58 – 00:12:00]** But so you can then compare the target state to the robot state **[00:12:00 – 00:12:05]** again, define rewards, and use reinforcement learning to update **[00:12:05 – 00:12:10]** those weights, While you freeze, basically, the control policies. **[00:12:10 – 00:12:13]** Again, it's important to always randomize your commands, **[00:12:13 – 00:12:15]** otherwise your system is not going to be robust. **[00:12:15 – 00:12:17]** That's where robustness comes from. **[00:12:17 – 00:12:21]** So, for example, you randomly choose waypoints that **[00:12:21 – 00:12:23]** you want to reach. **[00:12:23 – 00:12:25]** So let me show an example of this in action. **[00:12:25 – 00:12:29]** So here you see druids that learn to navigate. **[00:12:29 – 00:12:33]** You see the sensor simulated, and they give these target **[00:12:33 – 00:12:35]** points, the green points, and they learn basically to **[00:12:35 – 00:12:40]** get to these target points without bumping into any obstacles. **[00:12:40 – 00:12:43]** And so we do that by also procedurally changing the **[00:12:43 – 00:12:47]** obstacles or make it really challenging environments. **[00:12:47 – 00:12:51]** And so we also brought this to our robots of use. **[00:12:51 – 00:12:53]** So everything that you see here runs on a Jetson on **[00:12:53 – 00:12:54]** board of the robot. **[00:12:54 – 00:12:59]** We stream this video data that you see in the inset. **[00:12:59 – 00:13:02]** And the robot, online, does mapping, so it can map the **[00:13:02 – 00:13:05]** environment, it sees obstacles. **[00:13:05 – 00:13:09]** So if you didn't explore an area yet, it's an explorer droid, right? **[00:13:09 – 00:13:12]** Then you see white, but as soon as you see obstacles, it updates **[00:13:12 – 00:13:16]** the map and, you know, every voxel also has a confidence value, how **[00:13:16 – 00:13:21]** confident the robot is that there is an obstacle or no obstacle. **[00:13:21 – 00:13:26]** And so you can challenge the robots with these obstacles, **[00:13:26 – 00:13:31]** so that's Dario and Sami working on this part of the project. **[00:13:31 – 00:13:33]** And so you can make it really challenging and you see that **[00:13:33 – 00:13:37]** the robot learns to not bump into anything, it always goes **[00:13:37 – 00:13:41]** around the obstacles, and it does that in a quite natural way. **[00:13:41 – 00:13:44]** But the robot also does usually this scanning every now and **[00:13:44 – 00:13:47]** then, that's part of the story, it's an explorer droid, so things **[00:13:47 – 00:13:50]** like that are important to us. **[00:13:50 – 00:13:54]** All right, so here is another autonomy task **[00:13:54 – 00:14:00]** for us, human-robot interaction or guest-robot interaction. **[00:14:00 – 00:14:03]** And so here we had an amazing operator who really knows how to **[00:14:03 – 00:14:06]** get the best out of the character. **[00:14:06 – 00:14:10]** So how can you get to human-operator level autonomy with **[00:14:10 – 00:14:13]** a completely autonomous solution? **[00:14:13 – 00:14:14]** That's a huge challenge. **[00:14:14 – 00:14:17]** And I think that's a challenge in robotics in general, not **[00:14:17 – 00:14:20]** just for robotic characters Those are our use case. **[00:14:20 – 00:14:26]** So what you can do is we can record sensor data together **[00:14:26 – 00:14:29]** before an operator does. **[00:14:29 – 00:14:34]** That's a hugely important and valuable data set **[00:14:34 – 00:14:36]** that it can capture. **[00:14:36 – 00:14:39]** And so if you have these data sets captured, you **[00:14:39 – 00:14:41]** can basically train a network. **[00:14:41 – 00:14:45]** We use a diffusion model at this moment in time to imitate **[00:14:45 – 00:14:47]** what the operator does. **[00:14:47 – 00:14:53]** So you can then go and map sensor states to operator commands. **[00:14:53 – 00:14:56]** And so that's how we can actually achieve quite believable autonomy. **[00:14:56 – 00:15:02]** That's shown here for a robot that is shy, so we ask operators **[00:15:02 – 00:15:05]** to author content for a shy robot. **[00:15:05 – 00:15:09]** So it doesn't, it does never look towards Sammy, it tries **[00:15:09 – 00:15:12]** to look away, it says go away. **[00:15:12 – 00:15:16]** So it's remarkable how well this works, and you know, you need only **[00:15:17 – 00:15:21]** very little data for that, so not a huge, huge data set, so that was **[00:15:21 – 00:15:23]** quite interesting for us to see. **[00:15:23 – 00:15:28]** So really the personality comes out. **[00:15:28 – 00:15:32]** Alright, so the BDX druids were special as characters **[00:15:32 – 00:15:36]** because they took first steps in a park and now are going to appear **[00:15:36 – 00:15:38]** in the Mandalorian Grogu movie. **[00:15:38 – 00:15:40]** So usually if you go the other way around, we start in a **[00:15:41 – 00:15:45]** film or in a movie and bring these characters to life in the park. **[00:15:45 – 00:15:48]** And so that's actually a simpler problem to solve because robotic **[00:15:48 – 00:15:52]** characters, so we built literally a robotic character, a druid, **[00:15:52 – 00:15:56]** that's easier to build than trying to get other characters **[00:15:56 – 00:15:59]** to life in the physical world. **[00:16:00 – 00:16:02]** So, then we started working on OLAF. **[00:16:02 – 00:16:04]** And OLAF is an animated character. **[00:16:04 – 00:16:07]** Animated characters can do very non-physical things. **[00:16:07 – 00:16:11]** They don't have to observe the laws of physics. **[00:16:11 – 00:16:15]** For example, OLAF has a gigantic head, right, and has only two **[00:16:15 – 00:16:17]** snowballs sticking out for walking. **[00:16:17 – 00:16:20]** So how can you achieve something like that in the physical world? **[00:16:20 – 00:16:23]** That was a huge challenge to us. **[00:16:23 – 00:16:26]** At the beginning, I was actually pretty worried that we won't **[00:16:26 – 00:16:30]** succeed in building OLAF so that it's a high-quality, **[00:16:30 – 00:16:32]** Disney-quality experience. **[00:16:32 – 00:16:34]** But I think we got there. **[00:16:34 – 00:16:36]** And we got there within months. **[00:16:36 – 00:16:41]** So here you see OLAF taking steps at the World of Frozen in Paris. **[00:16:41 – 00:16:43]** This is gonna open up next week. **[00:16:44 – 00:16:46]** So you'll be able to see OLAF at the park **[00:16:46 – 00:16:49]** at Disneyland Paris next week. **[00:16:49 – 00:16:53]** So that was an amazing experience and it's a fantastic world **[00:16:53 – 00:16:57]** also for OLAF to walk around. **[00:16:57 – 00:17:02]** And so we brought, or are going to bring, OLAF to Disneyland Paris **[00:17:02 – 00:17:05]** and then soon after to also our Hong Kong park where there is an **[00:17:05 – 00:17:09]** existing World of Frozen already. **[00:17:09 – 00:17:11]** And so now I'm going to introduce you to a very **[00:17:11 – 00:17:16]** special guest, OLAF himself. **[00:17:16 – 00:17:22]** Hi, I'm OLAF and I like warm welcomes. **[00:17:25 – 00:17:28]** Good morning. **[00:17:31 – 00:17:34]** Look at us, out on the town. **[00:17:34 – 00:17:37]** I'm wearing my fancy kohl. **[00:17:37 – 00:17:39]** Yeah, also I iron my shirt, Olaf. **[00:17:39 – 00:17:41]** So you're quite a celebrity by now, huh? **[00:17:41 – 00:17:44]** Alright, nothing to see here. **[00:17:44 – 00:17:48]** Just an enchanted snowman walking among you. **[00:17:48 – 00:17:53]** So what has your favorite moment been at GTC so far? **[00:17:53 – 00:17:54]** Good question. **[00:17:54 – 00:17:59]** Um, maybe, nope, I can't narrow it down. **[00:17:59 – 00:18:04]** It's like asking me to choose my favorite snowflake. **[00:18:04 – 00:18:07]** So do you have any advice for us before we continue **[00:18:07 – 00:18:08]** with the presentation? **[00:18:08 – 00:18:10]** Do every possible thing to make friends with every single **[00:18:11 – 00:18:13]** person you meet the entire day. **[00:18:13 – 00:18:14]** Are you taking notes? **[00:18:14 – 00:18:19]** Yeah, noted. I'm gonna follow your advice, OLAF. **[00:18:19 – 00:18:21]** So thanks so much. We'll continue with the presentation. **[00:18:21 – 00:18:23]** You can come back later. **[00:18:23 – 00:18:24]** Goodbye, friends. **[00:18:24 – 00:18:28]** I'll never forget most of you. **[00:18:33 – 00:18:40]** Now, now, no need for applause, but no need for not applause either. **[00:18:42 – 00:18:45]** Thank you. Thank you. **[00:18:48 – 00:18:53]** Alright, now it's difficult to continue because you will **[00:18:53 – 00:18:56]** probably want to see OLAF more, but I'm gonna try. **[00:18:56 – 00:18:58]** Bye for now! **[00:18:59 – 00:19:03]** So here you see the mechatronics design, we can see inside **[00:19:03 – 00:19:07]** of OLAF and we actually use two legs that are exactly the same and **[00:19:07 – 00:19:12]** mounted them in opposite, you know, mounted them so that one knee bends **[00:19:12 – 00:19:15]** forward, the other one backward. **[00:19:15 – 00:19:19]** So for the arms we used so-called spherical linkages and also **[00:19:19 – 00:19:21]** in the head we used linkages. **[00:19:21 – 00:19:24]** And, you know, traditionally if you look at humanoid **[00:19:24 – 00:19:27]** characters you usually have actuators at the joints. **[00:19:27 – 00:19:31]** And so those systems are really hard to control. **[00:19:31 – 00:19:34]** So there is innovation in the leg design to maximize **[00:19:34 – 00:19:38]** the workspace, but then also we used many linkages to place **[00:19:38 – 00:19:41]** actuators where there is space. **[00:19:41 – 00:19:44]** For example, for the arms, we placed the actuators, two actuators **[00:19:44 – 00:19:49]** inside, and then the spherical linkage allowed us to really, **[00:19:49 – 00:19:52]** you know, move these like sticks that stick out of the snowman **[00:19:53 – 00:19:54]** without you seeing the actuators. **[00:19:54 – 00:19:57]** So that was a huge challenge for us. **[00:19:57 – 00:20:00]** So we use actually the same modular hardware components **[00:20:00 – 00:20:02]** like we used for BDX largely. **[00:20:02 – 00:20:06]** We added a couple of additional actuators to the actuator family, **[00:20:06 – 00:20:10]** characterized them, so we have good simulation representations, but **[00:20:10 – 00:20:12]** we used largely the same hardware. **[00:20:12 – 00:20:15]** And again, a Jetson is on board, we also have a battery on board that **[00:20:15 – 00:20:21]** lasts actually a bit less long than for the BDX druids, but long enough **[00:20:21 – 00:20:24]** for doing several shows in a row. **[00:20:24 – 00:20:26]** And show functions, if you have speakers on board, you **[00:20:26 – 00:20:31]** can also plug in to audio systems, as you heard, directly stream **[00:20:31 – 00:20:34]** the audio from the robot itself. **[00:20:34 – 00:20:37]** So model hardware is important, and for reinforcement learning, **[00:20:37 – 00:20:41]** we actually use this framework that I explained at the very **[00:20:41 – 00:20:43]** beginning for the BDX-Druids. **[00:20:43 – 00:20:47]** But there was a challenge, and the challenge was simulation, **[00:20:47 – 00:20:50]** or e-simulation in this context. **[00:20:50 – 00:20:51]** Why is this challenging? **[00:20:51 – 00:20:55]** Well, we have good simulators out there for articulated body systems. **[00:20:55 – 00:20:58]** Most simulators that are fast, GPU-accelerated, are made **[00:20:59 – 00:21:01]** for articulated body systems. **[00:21:02 – 00:21:05]** But so OLAF has these spherical linkages, has **[00:21:05 – 00:21:07]** also linkages in his head. **[00:21:07 – 00:21:11]** So you see the articulated body part here in green and the closed **[00:21:11 – 00:21:14]** loops or these linkages in blue. **[00:21:14 – 00:21:17]** And so the linkages are challenging to control with **[00:21:17 – 00:21:20]** control policy or simulate. **[00:21:20 – 00:21:24]** So we started working on a simulator. **[00:21:24 – 00:21:27]** So most simulators that you can use for reinforcement **[00:21:27 – 00:21:32]** learning, again, they interface well with articulated body systems. **[00:21:32 – 00:21:35]** Sometimes you have loop-close constraints that it can add, **[00:21:35 – 00:21:38]** but the simulators are made for more articulated body systems. **[00:21:38 – 00:21:39]** That's just not enough. **[00:21:39 – 00:21:42]** But the simulators are good on the GPUs. **[00:21:42 – 00:21:46]** For example, Isaac Sim is a good simulator that we've **[00:21:46 – 00:21:49]** used for the BDX druids. **[00:21:49 – 00:21:50]** And they support contact also. **[00:21:50 – 00:21:54]** Contact is important for walking characters or characters or **[00:21:54 – 00:21:58]** robots that manipulate the objects. **[00:21:58 – 00:22:02]** But then, so we have built expertise at DCM Research to **[00:22:02 – 00:22:07]** build simulators that can simulate so-called audio animatronics well. **[00:22:07 – 00:22:10]** And so systems are highly complex systems that have **[00:22:10 – 00:22:15]** many closed loops and use, you know, all sorts of joint types. **[00:22:15 – 00:22:18]** And so this is again to place actuators where there's space **[00:22:18 – 00:22:20]** and remote actuators. **[00:22:20 – 00:22:25]** And so for this particular animatronic fear that is in **[00:22:25 – 00:22:29]** E1000, we have 45 so-called loops. **[00:22:29 – 00:22:32]** And these are highly complex systems that existing simulators **[00:22:32 – 00:22:35]** struggle to simulate. **[00:22:35 – 00:22:38]** So we had these simulators, but they were targeting the **[00:22:38 – 00:22:42]** CPU, single instances, and we didn't have contact because **[00:22:42 – 00:22:46]** usually these systems act on stage. **[00:22:46 – 00:22:49]** And so then we started, as part of the Newton framework, **[00:22:49 – 00:22:54]** the collaboration with NVIDIA and Google DeepMind, we started working **[00:22:54 – 00:22:57]** on a Camino simulator that brings these two worlds together, supports **[00:22:57 – 00:23:00]** these more general mechanisms with these closed-loop systems, and does **[00:23:00 – 00:23:08]** this on a fundamental level, is GPU-accelerated and allows contact. **[00:23:09 – 00:23:13]** So the name Camino comes from the Star Wars planet, where the **[00:23:13 – 00:23:15]** inhabitants are experts in cloning. **[00:23:15 – 00:23:18]** So you clone many robots in simulation, that's why we **[00:23:18 – 00:23:21]** thought that name makes sense. **[00:23:22 – 00:23:25]** And again, the Camino simulator, the ultimate goal of that **[00:23:25 – 00:23:28]** simulator is to be able to apply reinforcement learning **[00:23:28 – 00:23:30]** to control any mechanism. **[00:23:30 – 00:23:32]** So that's the goal. **[00:23:33 – 00:23:34]** Very complex mechanisms, too. **[00:23:34 – 00:23:37]** And OLAF was just the first one we are going to use **[00:23:37 – 00:23:40]** this simulator for. **[00:23:40 – 00:23:43]** So if you want to learn more about Newton, our collaboration **[00:23:43 – 00:23:46]** with NVIDIA and DeepMind and Dear Progress, together with **[00:23:46 – 00:23:52]** our progress on Camino, come to this session on Newton on Thursday. **[00:23:52 – 00:23:56]** So we'll do a more deep dive there. **[00:23:56 – 00:23:59]** So ultimately, once you have a simulator, you can apply **[00:23:59 – 00:24:03]** reinforcement learning and follow these artistic motions well. **[00:24:05 – 00:24:08]** And so you can really get close to these artistic intents. **[00:24:08 – 00:24:12]** So you see the animation reference in the inset video here and **[00:24:12 – 00:24:17]** how close you can get with the two snowballs for walking. **[00:24:17 – 00:24:19]** So here is another reference animation. **[00:24:19 – 00:24:24]** And so again, the match between what's physically happening **[00:24:24 – 00:24:28]** to what the artist wanted, it's a perfect match. **[00:24:28 – 00:24:31]** You can't see the differences any longer. **[00:24:32 – 00:24:36]** Right, so one important aspect was also heel-toe walking. **[00:24:36 – 00:24:39]** So usually when you see robots walking, they always have **[00:24:39 – 00:24:43]** full contact with the ground, with the full surface of the foot. **[00:24:43 – 00:24:45]** So that didn't work for us. **[00:24:45 – 00:24:48]** Rolling off was actually quite important to do this or hit **[00:24:49 – 00:24:50]** the creative vision for this. **[00:24:50 – 00:24:54]** So we extended our tools on the procedural animation side **[00:24:54 – 00:24:57]** to achieve this goal. **[00:24:58 – 00:25:00]** And so we didn't stop innovating there. **[00:25:00 – 00:25:04]** So reinforcement learning provided us with control policies. **[00:25:04 – 00:25:09]** We were able to do first walking cycles on the physical system. **[00:25:09 – 00:25:14]** But we realized that this OLAF makes a lot of noise, too **[00:25:14 – 00:25:17]** much noise, when he walks around. **[00:25:17 – 00:25:19]** So that doesn't work, right? **[00:25:19 – 00:25:24]** The snowman doesn't do noisy walking, so we had to find **[00:25:24 – 00:25:25]** a solution for that. **[00:25:25 – 00:25:27]** And so I'm going to show you a video and I'll talk **[00:25:27 – 00:25:32]** about that video once the audio is fully played. **[00:25:37 – 00:25:40]** Too loud for snowmen. **[00:25:43 – 00:25:46]** So what we did here is we introduced so-called impact **[00:25:46 – 00:25:50]** reduction rewards so they really you know reduced the **[00:25:50 – 00:25:54]** noise that they can hear and that made a huge difference so **[00:25:54 – 00:25:59]** the second motion or the second you know video that I showed is much **[00:25:59 – 00:26:03]** much better much more believable and so if I switched off the audio **[00:26:04 – 00:26:07]** you would not see a difference it's a much more believable **[00:26:07 – 00:26:12]** motion and impact reduction is useful in general for robotics **[00:26:12 – 00:26:16]** Because if you can reduce the impact forces that act **[00:26:16 – 00:26:20]** on a mechanical system, you have less wear on the system, **[00:26:20 – 00:26:24]** especially also on your actuators, and you have to replace them less. **[00:26:24 – 00:26:27]** So it actually helps to reduce maintenance cost, or has the **[00:26:27 – 00:26:29]** potential to do so. **[00:26:29 – 00:26:33]** So what we innovated for OLAF has applications elsewhere as well. **[00:26:34 – 00:26:37]** And then the second innovation was that we introduced rewards **[00:26:37 – 00:26:43]** that help with, basically, OLAF learning to not melt. **[00:26:43 – 00:26:47]** So he loves summer, so we need to prevent him from melting. **[00:26:47 – 00:26:52]** And so we saw overheating for the actuators. **[00:26:52 – 00:26:55]** So we introduced thermal dynamics modeling. **[00:26:55 – 00:26:58]** So we not only simulate the mechanical system, but also the **[00:26:58 – 00:27:02]** thermal dynamics of the actuators to prevent overheating, make the **[00:27:02 – 00:27:07]** least amount of changes possible to stay as close as possible to the **[00:27:07 – 00:27:11]** artistic intent while preventing the system from failing due **[00:27:11 – 00:27:14]** to overheating in the actuators. **[00:27:14 – 00:27:17]** So we have additional rewards that help with that. **[00:27:17 – 00:27:21]** And so what I'm going to show you is two videos side by side. **[00:27:21 – 00:27:24]** And so OLAF has a huge head. **[00:27:24 – 00:27:28]** And the problem here is we had to fit three small actuators. **[00:27:28 – 00:27:31]** So you have all the degrees of freedom necessary to move **[00:27:31 – 00:27:33]** the head in the thin neck. **[00:27:33 – 00:27:36]** We didn't want to make any changes because it's OLAF, you can't. **[00:27:36 – 00:27:38]** You have to keep the outside the same. **[00:27:38 – 00:27:42]** So large head, even if you reduce the weights as much **[00:27:42 – 00:27:46]** as possible, we saw overheating of those actuators in the neck. **[00:27:46 – 00:27:50]** And so with those additional rewards and the thermal dynamics **[00:27:50 – 00:27:54]** modeling, we could prevent basically the actuators to **[00:27:54 – 00:28:01]** ever cross a certain temperature limit with a safety margin. **[00:28:01 – 00:28:03]** And so I'm going to show these two videos. **[00:28:03 – 00:28:06]** And so what you are going to see with thermal modeling **[00:28:06 – 00:28:12]** is that the higher the temperature, **[00:28:13 – 00:28:17]** The more we lean forward, and if the temperature is **[00:28:17 – 00:28:21]** decreasing again, the robot goes back and leans backward. **[00:28:21 – 00:28:24]** So, very subtle changes. **[00:28:24 – 00:28:27]** An operator would not perceive that, especially a guest would **[00:28:27 – 00:28:27]** not perceive that. **[00:28:27 – 00:28:30]** You would not perceive these differences. **[00:28:30 – 00:28:35]** But OLAF literally learned to manage its own heat. **[00:28:35 – 00:28:38]** And I think that's huge also in general for robotics. **[00:28:38 – 00:28:43]** Think of like robots working outdoors in outdoor environments **[00:28:43 – 00:28:46]** where it's really hot outside, so thermal modeling will **[00:28:46 – 00:28:49]** become more important. **[00:28:49 – 00:28:51]** Okay, so I want to briefly give you an update on what **[00:28:51 – 00:28:52]** we do on the research side. **[00:28:52 – 00:28:56]** We've built this humanoid character for research purposes. **[00:28:56 – 00:29:01]** And recently, we looked actually into soft and stylized falling. **[00:29:01 – 00:29:05]** And so falling is always perceived as a failure. **[00:29:05 – 00:29:08]** I don't think falling is a failure. **[00:29:08 – 00:29:10]** I think you should not exclude that from your control stack. **[00:29:10 – 00:29:14]** You should include it, because we can detect the fall and **[00:29:14 – 00:29:18]** then fall in a natural way, and in a way that doesn't **[00:29:18 – 00:29:20]** have an impact on your components. **[00:29:21 – 00:29:24]** So what you're trying to do here with a policy is **[00:29:24 – 00:29:28]** we trained a policy where you can provide a target **[00:29:28 – 00:29:30]** pose the robot should fall into. **[00:29:30 – 00:29:35]** And we had rewards to reduce the impact forces that act **[00:29:35 – 00:29:40]** onto the components so you can prevent components from failing. **[00:29:40 – 00:29:43]** So stylized falling helps also if you have a target **[00:29:43 – 00:29:48]** post the robot can fall into, you can also do more easy recovery **[00:29:48 – 00:29:50]** afterwards and in-style recovery. **[00:29:50 – 00:29:53]** So I think it's something important for robots in general, **[00:29:53 – 00:29:57]** we need to embrace that falling is just part of functionality **[00:29:57 – 00:29:59]** that we need to cover. **[00:29:59 – 00:30:00]** And it's not a failure. **[00:30:00 – 00:30:04]** So we consider that part of the character. **[00:30:05 – 00:30:08]** So here it's quite remarkable how, in a very short amount **[00:30:08 – 00:30:12]** of time, if you train a policy well enough, the robot can react to that **[00:30:12 – 00:30:14]** and fall into a particular pose. **[00:30:14 – 00:30:17]** It can even turn itself. **[00:30:24 – 00:30:28]** Okay, you can tell that we are having fun every day. **[00:30:28 – 00:30:31]** Even if the robots don't look beautiful, it's still quite **[00:30:31 – 00:30:33]** funny to work with them. **[00:30:33 – 00:30:36]** Okay, so now autonomy, how does the future look like? **[00:30:36 – 00:30:40]** Well, we have these MULTIMODAL reasoning models or MULTIMODAL **[00:30:40 – 00:30:43]** large language models that can do long horizon planning **[00:30:43 – 00:30:46]** for us and can provide us with high level commands as **[00:30:46 – 00:30:49]** an input for the next steps. **[00:30:49 – 00:30:52]** But the question is, how can you interface with those high-level **[00:30:52 – 00:30:56]** commands and how can you bring this to a robotic character **[00:30:56 – 00:30:58]** or a robot in general? **[00:30:58 – 00:31:01]** And so here we believe that this is a two-step process. **[00:31:01 – 00:31:04]** So motion diffusion models are quite useful. **[00:31:05 – 00:31:08]** We use self-supervised learning to extract from large data **[00:31:08 – 00:31:13]** sets, motion data sets, the structure to understand the **[00:31:13 – 00:31:15]** structure, understand motion. **[00:31:15 – 00:31:19]** And so video data is actually quite helpful for scalability. **[00:31:19 – 00:31:23]** In videos, you can actually observe motion, extract motion. **[00:31:23 – 00:31:26]** But so there is one bottleneck with that, and I'll talk about **[00:31:26 – 00:31:29]** that right after showing what a diffusion model does. **[00:31:29 – 00:31:31]** So a diffusion model can take high-level commands **[00:31:31 – 00:31:35]** as a conditional input, takes noise also, and denoises. **[00:31:35 – 00:31:40]** And usually we have seen that is for images, but here is **[00:31:40 – 00:31:44]** a visualization of that process, the denoising process on a robot. **[00:31:44 – 00:31:47]** And in the end, you see a very, very smooth motion. **[00:31:47 – 00:31:50]** So you could have a conditional input saying **[00:31:50 – 00:31:53]** I want this robot to do **[00:31:53 – 00:31:56]** You know, Michael Jackson moonwalk, and then with that **[00:31:56 – 00:31:59]** condition, it could perform **[00:31:59 – 00:32:06]** that motion based on learning from large data sets of motions. **[00:32:06 – 00:32:08]** But the question is, this is not physics-informed, this **[00:32:08 – 00:32:10]** is kinematic motion. **[00:32:10 – 00:32:13]** And so kinematic motion needs to somehow be mapped to the **[00:32:13 – 00:32:18]** robot itself, and for that we need reinforcement learning, **[00:32:18 – 00:32:21]** we need simulation, because what it can't observe in video **[00:32:21 – 00:32:24]** data is forces and torques. **[00:32:24 – 00:32:27]** So you need to close the gap, and that's why simulation **[00:32:27 – 00:32:31]** and reinforcement learning is so incredibly important as a tool, and **[00:32:31 – 00:32:33]** will stay relevant in this context. **[00:32:33 – 00:32:36]** So we can train them with reinforcement learning based **[00:32:36 – 00:32:38]** on this large data set of motion. **[00:32:38 – 00:32:42]** We can train a tracking policy so we can basically close the loop **[00:32:42 – 00:32:45]** fully and bring this to characters. **[00:32:45 – 00:32:49]** We've demonstrated this already a while ago, these results. **[00:32:49 – 00:32:52]** We have some more exciting things in the pipeline there. **[00:32:52 – 00:32:56]** But so I think this could basically be a pipeline that can **[00:32:56 – 00:33:00]** scale quite well and do this soon. **[00:33:00 – 00:33:03]** Alright, so this is the team in Zurich, that's the team **[00:33:03 – 00:33:06]** I get to work with, I get to present here on stage, but it's **[00:33:06 – 00:33:10]** actually those team members that should be in the spotlight here. **[00:33:10 – 00:33:14]** Everyone has a unique expertise, and without your expertise, **[00:33:14 – 00:33:18]** robots like a BDX or OLAF would not be possible. **[00:33:18 – 00:33:22]** And you also have half of the team here, so I'm going **[00:33:22 – 00:33:23]** to call them out briefly. **[00:33:23 – 00:33:27]** Agon, Dario, David, Espen and Ruben, if you could get **[00:33:28 – 00:33:31]** up, stand up briefly. **[00:33:39 – 00:33:42]** So if you have questions that you rather should ask them, **[00:33:42 – 00:33:45]** they are the true experts. **[00:33:45 – 00:33:48]** And so we also get tremendous amount of help to bring these **[00:33:48 – 00:33:53]** characters to our parks, deploy them, productize them. **[00:33:53 – 00:33:56]** There's hardware productization, there's software productization. **[00:33:56 – 00:34:00]** So it is truly a family behind this all. **[00:34:00 – 00:34:03]** Many, many, many contributors to make this possible. **[00:34:03 – 00:34:07]** So I'm very grateful that we have this support system. **[00:34:07 – 00:34:10]** These are references if you are interested in reading **[00:34:10 – 00:34:16]** up on some of this technology. **[00:34:18 – 00:34:23]** And then come to our session on Newton simulation, because **[00:34:23 – 00:34:27]** with good simulators that balance accuracy with parallelism **[00:34:27 – 00:34:32]** and speed, you can not only achieve walking with these robotic **[00:34:32 – 00:34:34]** characters, but performances. **[00:34:34 – 00:34:39]** So here is a robot doing a pirouette. **[00:34:39 – 00:34:42]** And so it's incredibly hard because frictional contact **[00:34:42 – 00:34:44]** is hard to model properly. **[00:34:44 – 00:34:46]** And so you see here stick-slip contact. **[00:34:46 – 00:34:48]** So if you wanted to learn more about simulation **[00:34:48 – 00:34:51]** and what it can help with, especially in combination **[00:34:51 – 00:34:55]** with reinforcement learning, come to our session on Thursday. **[00:34:55 – 00:34:58]** And last but not least, we have a landing page for Camino. **[00:34:58 – 00:35:00]** Camino is available now. **[00:35:00 – 00:35:02]** You can try Camino. **[00:35:02 – 00:35:05]** So you can find this landing page online, and this gives **[00:35:05 – 00:35:08]** you pointers on how to use Camino. **[00:35:08 – 00:35:11]** It's in a beta state, but we wanted to release it to get **[00:35:11 – 00:35:14]** first feedback and for you to use. **[00:35:15 – 00:35:17]** So thanks so much. **[00:35:26 – 00:35:30]** So now OLAF is here to answer your questions. **[00:35:33 – 00:35:37]** We can take a few questions.. **[00:35:42 – 00:35:44]** and interactions. **[00:35:44 – 00:35:47]** You mentioned you don't need many data to train, let's **[00:35:47 – 00:35:48]** say, the shy robot. **[00:35:48 – 00:35:51]** So what's the process there to capture the emotions? **[00:35:51 – 00:35:54]** Yeah, data capture for interactions, human-robot **[00:35:54 – 00:35:57]** interaction in particular. **[00:35:57 – 00:36:00]** So the data capture, we simplified the sensors for now. **[00:36:00 – 00:36:04]** We used the mocap system to just get a sense of does that **[00:36:04 – 00:36:08]** scale at all, because we didn't know how much data you would need. **[00:36:08 – 00:36:11]** So we abstracted away the sensors with mocap. **[00:36:11 – 00:36:15]** So you saw that Sami, who worked on this project, was wearing a **[00:36:15 – 00:36:18]** hat with mocap markers on the head. **[00:36:18 – 00:36:24]** And so we didn't have full body mocap, so just on the head, **[00:36:24 – 00:36:29]** but that was actually sufficient to achieve quite good results already. **[00:36:29 – 00:36:33]** So we captured everything that an operator does, together **[00:36:33 – 00:36:38]** with mocap markers on the robot and on the human's head. **[00:36:44 – 00:36:47]** Once more, but louder. **[00:36:49 – 00:36:52]** Is there another question you could help answer? **[00:36:52 – 00:36:54]** Hello, good morning. My name is Jonathan Ramos. **[00:36:54 – 00:36:56]** I'm a GenAI team lead. **[00:36:56 – 00:36:58]** I have a question concerning the use of large language **[00:36:58 – 00:37:02]** models, particularly how did it simplify your workflow **[00:37:02 – 00:37:06]** on getting OLAF from A to B? **[00:37:06 – 00:37:08]** So, that's an excellent question. **[00:37:08 – 00:37:10]** Do you want to take this one, Olaf? **[00:37:11 – 00:37:14]** No, I'm not sure. **[00:37:15 – 00:37:19]** So at the moment OLAF is actually completely remote controlled and we **[00:37:19 – 00:37:24]** trigger recorded lines so you can actually hear Josh Gadd's voice. **[00:37:24 – 00:37:29]** So for OLAF that's going to stay that way. **[00:37:29 – 00:37:33]** But so for characters in general, large language models can **[00:37:33 – 00:37:37]** help to understand, especially with multimodal input. **[00:37:37 – 00:37:40]** Data can understand the world better, can do longer horizon **[00:37:40 – 00:37:44]** planning, and could basically help with our navigation tasks **[00:37:44 – 00:37:50]** because you need to somehow get from, you know, break down a larger **[00:37:50 – 00:37:55]** problem into tasks, subtasks, and then the navigation engine **[00:37:55 – 00:37:57]** could execute those subtasks. **[00:37:57 – 00:38:01]** For OLAF, we don't have any... **[00:38:02 – 00:38:05]** Any reasoning model, anytime soon, it's also, you know, you get **[00:38:05 – 00:38:13]** so much personality out if a human operator operates the... Thanks! **[00:38:15 – 00:38:20]** Is there one more question that I can answer? **[00:38:24 – 00:38:29]** Do you see in the future that character design will be based **[00:38:29 – 00:38:33]** more around manufacturability to bring those characters into the **[00:38:33 – 00:38:38]** parks as opposed to the character always coming first and then **[00:38:38 – 00:38:41]** the robot having to follow suit? **[00:38:42 – 00:38:46]** So manufacturability, basically optimizing robots **[00:38:46 – 00:38:49]** for manufacturability, that was the question. **[00:38:49 – 00:38:54]** So to some degree, if you've worked with modeler hardware components, **[00:38:54 – 00:38:56]** that already gets you a step there. **[00:38:56 – 00:39:00]** Because if you use production-grade actuators, production-grade **[00:39:00 – 00:39:03]** sensors, production-grade electronics inside, and their **[00:39:03 – 00:39:08]** modeler, you can use them across many characters. **[00:39:08 – 00:39:10]** That's basically already helping in the process. **[00:39:11 – 00:39:13]** But you probably think of components and **[00:39:13 – 00:39:14]** manufacturability of components. **[00:39:14 – 00:39:17]** And there are always a few refinement cycles that need **[00:39:17 – 00:39:21]** to happen to make a character truly production ready or **[00:39:21 – 00:39:23]** robot production ready. **[00:39:23 – 00:39:27]** So I think, you know, for us, manufacturability, we **[00:39:27 – 00:39:29]** don't do that at scale as much. **[00:39:29 – 00:39:31]** We customize characters. **[00:39:31 – 00:39:33]** We build only a few OLAFs. **[00:39:33 – 00:39:37]** You never will see, other than in a simulation video, **[00:39:37 – 00:39:39]** several OLAFs on a stage. **[00:39:40 – 00:39:45]** So we optimize basically for fewer characters but building **[00:39:45 – 00:39:46]** custom characters more quickly. **[00:39:46 – 00:39:50]** So manufacturability is an important item for us but **[00:39:50 – 00:39:55]** maybe for our use case not the most important one. **[00:40:02 – 00:40:06]** Shall we take another question? **[00:40:06 – 00:40:09]** We're technically at time, so we should say goodbye. **[00:40:09 – 00:40:11]** Sorry, my ears were icy. **[00:40:11 – 00:40:14]** Say that again? **[00:40:14 – 00:40:16]** Should say goodbye. **[00:40:16 – 00:40:17]** Goodbye, friends. **[00:40:17 – 00:40:22]** I'll never forget most of you. **[00:40:22 – 00:40:25]** So thanks so much for coming. **[00:40:26 – 00:40:28]** Absolutely. **[00:40:28 – 00:40:29]** Oh, thank you.