# Developer Breakout: A New Era of Advanced Physics Simulation

## Abstract

Meet the pioneers of NVIDIA physics simulation technologies. Join this breakout session with the lead engineers and product managers of NVIDIA PhysX, Flow, and Blast where you can discuss various simulation topics, get answers to your questions, and learn what's next on their list to tackle.

## AI Summary

_No AI summary available._

## Transcript

**[00:00:01 – 00:00:04]** Sorry about that, some technical issue in there.

**[00:00:04 – 00:00:07]** So I hope you heard what I said.

**[00:00:07 – 00:00:12]** So now maybe Alesh can go to introduce himself.

**[00:00:13 – 00:00:14]** Hi, I'm Alesh.

**[00:00:14 – 00:00:18]** I'm working on the Omniverse Physics integrations, which

**[00:00:18 – 00:00:25]** is taking care of the USD and physics, putting things together.

**[00:00:25 – 00:00:28]** Let me give the ball to Andrew.

**[00:00:29 – 00:00:33]** Hi, I'm Andrew Reidmeyer, lead developer on Flow,

**[00:00:33 – 00:00:34]** doing smoke and fire.

**[00:00:34 – 00:00:36]** I'm Kier.

**[00:00:36 – 00:00:39]** Hi, I'm Kier.

**[00:00:39 – 00:00:43]** I'm the lead programmer on the PhysX SDK, and I work on various

**[00:00:43 – 00:00:48]** things like rigid-body dynamics, soft bodies, fluids, particles,

**[00:00:49 – 00:00:53]** articulations for robotics, and various things like that.

**[00:00:54 – 00:00:56]** Hi, I'm Michael Hapala.

**[00:00:56 – 00:00:58]** I joined NVIDIA about two years ago and I'm focusing

**[00:00:58 – 00:01:02]** mostly on Omniverse Physics UI.

**[00:01:03 – 00:01:08]** Okay, so let me show you some amazing tech we did

**[00:01:08 – 00:01:11]** with 103 release, right?

**[00:01:11 – 00:01:14]** So let's look at one of the video.

**[00:01:14 – 00:01:18]** So this is called Warehouse Horror.

**[00:01:18 – 00:01:21]** So I hope you enjoy it.

**[00:01:50 – 00:01:52]** Okay, maybe we should talk about some of the techniques

**[00:01:52 – 00:01:54]** we use in this video, right.

**[00:01:54 – 00:02:00]** So, you see in this video, we got maybe thousands of

**[00:02:01 – 00:02:02]** rigid bodies in there, right.

**[00:02:02 – 00:02:06]** We're using convert decomposition to...

**[00:02:10 – 00:02:14]** And also, one of the selling points of this video is the

**[00:02:14 – 00:02:17]** new force field feature.

**[00:02:22 – 00:02:25]** Sorry, for some reason, I think somebody keeps hitting

**[00:02:25 – 00:02:30]** play or something and it keeps interfering with your talking.

**[00:02:30 – 00:02:33]** I know, I don't.

**[00:02:33 – 00:02:35]** Sorry, please don't touch anything.

**[00:02:35 – 00:02:38]** Let me show another video.

**[00:02:39 – 00:02:41]** We can show this one first,

**[00:02:42 – 00:02:45]** so I'm going to the file and then let's look at the Franka,

**[00:02:45 – 00:02:50]** Luk and Bo close-up video.

**[00:04:00 – 00:04:06]** Maybe Keir can explain the technique used behind this scene?

**[00:04:06 – 00:04:07]** Sure, yeah.

**[00:04:07 – 00:04:11]** So this is using STF collision.

**[00:04:16 – 00:04:18]** I'm sorry, I don't know what happened.

**[00:04:18 – 00:04:20]** I think we have a ghost in the machine somewhere.

**[00:04:20 – 00:04:22]** It suddenly jumped out.

**[00:04:22 – 00:04:24]** I promise I didn't touch anything.

**[00:04:24 – 00:04:26]** Yeah, hands up for everyone.

**[00:04:26 – 00:04:29]** Yeah, so what we saw before was using SDF collision, so

**[00:04:29 – 00:04:32]** it was using mesh-mesh collision representing the nuts and the

**[00:04:32 – 00:04:35]** bolts using signed distance fields, and then using a new feature

**[00:04:35 – 00:04:40]** that's in PhysX 5.1 that allows us to be able to do high detail.

**[00:04:40 – 00:04:43]** Here we go again.

**[00:04:44 – 00:04:47]** I have to keep canceling, I don't really understand why,

**[00:04:47 – 00:04:51]** but sorry, Kia, please continue.

**[00:04:52 – 00:04:53]** Yep, so it's using these signed distance fields to

**[00:04:53 – 00:04:57]** be able to do collision between these detailed meshes.

**[00:05:05 – 00:05:07]** There's definitely somebody who's figured out how to interfere

**[00:05:07 – 00:05:09]** with this session and they're having a lot of fun.

**[00:05:09 – 00:05:14]** Of course, in the machine, right?

**[00:05:14 – 00:05:16]** Let's continue, yeah.

**[00:05:16 – 00:05:18]** Okay, so with the assigned distance field collision,

**[00:05:18 – 00:05:21]** what we're able to do is we're able to generate fully accurate

**[00:05:21 – 00:05:25]** collisions between arbitrary meshes, which allows us to be able

**[00:05:25 – 00:05:29]** to simulate complex interactions between these nuts and the bolts.

**[00:05:29 – 00:05:30]** The bolt then

**[00:05:30 – 00:05:32]** Nuts are being fed through a vibratory feeder system,

**[00:05:32 – 00:05:35]** which basically the simulation is running at 240 Hz.

**[00:05:35 – 00:05:38]** It actually runs in real time.

**[00:05:38 – 00:05:41]** Simulation's been running at 240 Hz, the rendering's

**[00:05:41 – 00:05:42]** running at 60 Hz.

**[00:05:42 – 00:05:45]** This feeder mechanism is vibrating very fast, which actually

**[00:05:45 – 00:05:49]** causes the bolts to move down this channel to go to the gripper.

**[00:05:49 – 00:05:52]** The gripper then grabs the nut, picks it up, puts it

**[00:05:52 – 00:05:54]** on the bolt, screws it on.

**[00:05:54 – 00:05:55]** And this will be a real-time demo that

**[00:05:56 – 00:05:58]** will be released with Omniverse.

**[00:05:58 – 00:06:02]** physics 103 that allow you to be able to see this and interact

**[00:06:02 – 00:06:06]** with it and kind of uh see how this uh new feature that we're working

**[00:06:06 – 00:06:12]** on uh behaves yeah uh one thing to know right this is not a actual uh

**[00:06:12 – 00:06:16]** official uh portalization feature so this is still work in progress

**[00:06:16 – 00:06:23]** but uh because this feature is so powerful right we let the the let's

**[00:06:23 – 00:06:27]** say we have a robotic case The measure is extremely complicated.

**[00:06:27 – 00:06:31]** and we want to represent it as close as possible it's

**[00:06:31 – 00:06:34]** such a powerful tool and then we are able to run it in real

**[00:06:34 – 00:06:38]** time that's why we are shipping it in real demo let the user

**[00:06:38 – 00:06:42]** to play reset and then give us some feedback and then we can improve

**[00:06:42 – 00:06:47]** it and properly protocolize it in the future release okay so we got

**[00:06:47 – 00:06:50]** a question says i promise i didn't press the play button i believe

**[00:06:50 – 00:06:54]** you owen how many contacts are in the stf collision approximately

**[00:06:54 – 00:06:59]** And so it peaks at about 16,000 that it generates because

**[00:06:59 – 00:07:03]** it's clipping the face of the vault, so of the nut, which

**[00:07:03 – 00:07:09]** is about 16,000 features on the actual interface against the vault.

**[00:07:09 – 00:07:12]** What then ends up being simulated in the solver is a subset of that

**[00:07:12 – 00:07:15]** just simply because we then run a post-process that simplifies the

**[00:07:15 – 00:07:18]** mesh, it simplifies the contacts and throws away quite a lot of them

**[00:07:18 – 00:07:19]** before we put them into the solver.

**[00:07:19 – 00:07:22]** Otherwise we'd be basically overloading it with a massive

**[00:07:22 – 00:07:24]** number of redundant contacts.

**[00:07:24 – 00:07:28]** So it winds up settling down at somewhere between about 200 and

**[00:07:28 – 00:07:31]** 300 contacts that make it through to the actual simulation when it's

**[00:07:32 – 00:07:35]** simulating them screwing together.

**[00:07:35 – 00:07:42]** Okay, let me play another video, right, to look at our amazing tech.

**[00:07:54 – 00:07:55]** This one is a Lego buggy, right?

**[00:07:55 – 00:07:59]** So in this scene...

**[00:08:44 – 00:08:48]** I don't know what's going on but it seems played a little

**[00:08:48 – 00:08:54]** bit and then someone stopped it or something so I'm yeah I

**[00:08:54 – 00:08:56]** think they're saying it in the chat that they have the impression that

**[00:08:57 – 00:08:59]** every listener is able to basically click and do whatever they

**[00:08:59 – 00:09:01]** want with these videos so See it?

**[00:09:01 – 00:09:04]** Allow it to come up by himself.

**[00:09:05 – 00:09:10]** This is obviously not working well.

**[00:09:10 – 00:09:11]** Please stop picking that.

**[00:09:11 – 00:09:15]** And then let me let us go through all the video, right?

**[00:09:15 – 00:09:18]** We promise if you want to come back and then we will show it again.

**[00:09:19 – 00:09:22]** So, uh, let me try again.

**[00:09:22 – 00:09:26]** So let's go through everything.

**[00:09:28 – 00:09:33]** So what I'm going to show is the Lego buggy video, right?

**[00:09:33 – 00:09:38]** So in that scene, someone is playing the...

**[00:11:14 – 00:11:17]** Great, thanks. Successfully, we can play this video.

**[00:11:17 – 00:11:23]** So in this video, the buggy is constructed with lots of

**[00:11:23 – 00:11:24]** rigid bodies, right?

**[00:11:24 – 00:11:30]** And each one is connected by joint, and we're using JoyDrive

**[00:11:30 – 00:11:35]** to simulate the suspension and motor, everything in the Lego car.

**[00:11:35 – 00:11:39]** And also in the video, you can see we have a few different camera

**[00:11:39 – 00:11:44]** aspect to showing you the mechanic actually simulated in the buggy.

**[00:11:45 – 00:11:47]** Yeah, there are actually two questions.

**[00:11:47 – 00:11:51]** One was of if the car is running with pure physics gears, etc.,

**[00:11:51 – 00:11:54]** and as far as I know, I mean, there's no animation, right?

**[00:11:54 – 00:11:56]** The whole thing is run only by pure physics.

**[00:11:56 – 00:11:59]** 100% physics, yes.

**[00:11:59 – 00:12:02]** And the second one is, can SDF-based collisions also

**[00:12:03 – 00:12:05]** be applied to soft bodies?

**[00:12:06 – 00:12:07]** They can be.

**[00:12:07 – 00:12:08]** That's not something that we've implemented inside

**[00:12:08 – 00:12:10]** Omniverse Physics yet.

**[00:12:10 – 00:12:12]** But they can very much, it's something that they

**[00:12:12 – 00:12:13]** definitely can be.

**[00:12:13 – 00:12:18]** At the moment we're using pure tetrahedral meshes for soft

**[00:12:18 – 00:12:21]** body collision, but using SDFs for them as a way to accelerate them

**[00:12:21 – 00:12:26]** is 100% on the cards as something that we'll do in the future.

**[00:12:27 – 00:12:32]** Yeah, clearly we want to make sure that Fisat SDK itself

**[00:12:32 – 00:12:36]** can support all the features that have SDF, right?

**[00:12:36 – 00:12:42]** Because we recognize the importance of SDF for support as well.

**[00:12:42 – 00:12:46]** That will be on our future release.

**[00:12:47 – 00:12:49]** OK, let me show you another one.

**[00:12:49 – 00:12:51]** OK, just before you go to another one, there are still a

**[00:12:51 – 00:12:53]** couple of questions for the buggy.

**[00:12:53 – 00:12:56]** So there are a couple about sound.

**[00:12:56 – 00:12:58]** So if the engine sound is generated from some kind of

**[00:12:58 – 00:13:01]** ML or is it computed from physics?

**[00:13:01 – 00:13:06]** And then a second one connected to it, is it synthesized in real time?

**[00:13:06 – 00:13:10]** yes it's basically synthesized based on the on the velocity

**[00:13:10 – 00:13:17]** vector read it from the vehicle rigid body so yeah it just scales

**[00:13:17 – 00:13:25]** up the sound complexity basically based on the current velocity

**[00:13:25 – 00:13:29]** all right and then the last one for the buggy is does the buggy being

**[00:13:29 – 00:13:34]** in pure physics affect the fps in any way well i guess it does but

**[00:13:34 – 00:13:39]** Um, well, I mean, it's more expensive to simulate like

**[00:13:39 – 00:13:42]** around, I think it's around 130 different degrees of freedom

**[00:13:42 – 00:13:43]** joints that are in there.

**[00:13:43 – 00:13:46]** Uh, but no, it's, it's not particularly expensive.

**[00:13:46 – 00:13:50]** I mean, we can run that on both CPU or GPU and it's able to run

**[00:13:50 – 00:13:52]** it, uh, at a high, high frame rate.

**[00:13:53 – 00:13:55]** So it's, it's actually the underlying simulation and

**[00:13:55 – 00:13:57]** that is running at 240 Hertz and you could still solidly

**[00:13:57 – 00:14:00]** get 60 frames a second.

**[00:14:00 – 00:14:03]** Not only that, but you could have, when using GPU, you

**[00:14:03 – 00:14:08]** could have tens, hundreds, even thousands of those driving around.

**[00:14:08 – 00:14:10]** So it's not dramatically expensive.

**[00:14:10 – 00:14:13]** In fact, we've done some things where we juice it up a little

**[00:14:13 – 00:14:17]** bit more and make the tires.

**[00:14:17 – 00:14:20]** Use soft bodies, which I think we might have a video.

**[00:14:20 – 00:14:23]** I don't know, do we have a video of that, Michelle, or?

**[00:14:23 – 00:14:25]** No, we don't have that yet.

**[00:14:25 – 00:14:28]** The reason for that is because there are still some issue

**[00:14:28 – 00:14:31]** with the soft body tie at the version, right?

**[00:14:31 – 00:14:38]** Because we need to make sure the tie is not a lot more steep, right?

**[00:14:38 – 00:14:42]** If that is the case, we need to run a little bit more iteration.

**[00:14:42 – 00:14:44]** So we're working on how to improve the performance and

**[00:14:44 – 00:14:46]** stability of the tie at the moment.

**[00:14:46 – 00:14:50]** So I think that will come up in the next release.

**[00:14:50 – 00:14:53]** And there's also a question about the demos and if they

**[00:14:53 – 00:14:56]** will be available in Omniverse.

**[00:14:57 – 00:14:59]** I mean, in the next release, you'll see, I guess, most of

**[00:14:59 – 00:15:00]** the demos that we're showing here.

**[00:15:00 – 00:15:03]** Maybe all of them, even.

**[00:15:04 – 00:15:07]** Maybe a small note that they won't be enabled by default, that all

**[00:15:07 – 00:15:14]** these new features, all this cool stuff that is still a little bit

**[00:15:14 – 00:15:19]** in working progress is showcased in preview extension, so it's not part

**[00:15:19 – 00:15:23]** of the default release, you need to manually enable this extension.

**[00:15:23 – 00:15:29]** Yep, as Alex mentioned, right, you can enable the preview extension in

**[00:15:29 – 00:15:32]** Create to play with all this demo.

**[00:15:32 – 00:15:35]** Also, we have a showroom app coming as well.

**[00:15:35 – 00:15:38]** Inside have some really beautiful, we are proud

**[00:15:38 – 00:15:40]** of Ted and showing in there.

**[00:15:40 – 00:15:42]** Lux and Bose is one of it.

**[00:15:42 – 00:15:45]** Let me show you another one.

**[00:16:13 – 00:16:17]** Okay, so in this scene, it's clearly, we are showing a Franka

**[00:16:17 – 00:16:19]** arm and pick up a soft body, right?

**[00:16:19 – 00:16:26]** So, soft body in there, which is showcase how the rigid

**[00:16:26 – 00:16:31]** body and soft body interruption and we're using friction to

**[00:16:31 – 00:16:34]** lick up the soft body in there.

**[00:16:35 – 00:16:39]** Kia, do you have anything to say about this demo?

**[00:16:39 – 00:16:42]** I mean, not a lot, really. I mean, it's an articulated

**[00:16:42 – 00:16:44]** robot, the one that you're probably quite familiar with,

**[00:16:44 – 00:16:48]** called a Franka, using inverse kinematics to drive forwards to

**[00:16:48 – 00:16:51]** be able to grab these soft bodies.

**[00:16:51 – 00:16:55]** The soft bodies are simulated using a co-rotated FEM model,

**[00:16:55 – 00:17:00]** which has been augmented slightly to have better volume conservation.

**[00:17:00 – 00:17:02]** And yeah, it just works.

**[00:17:02 – 00:17:03]** It works really well.

**[00:17:03 – 00:17:06]** The slip is actually by design, just so we wanted to drop

**[00:17:06 – 00:17:07]** it and pick it back up.

**[00:17:07 – 00:17:09]** So it's actually

**[00:17:09 – 00:17:10]** There's a couple of questions in general.

**[00:17:10 – 00:17:14]** One is what hardware was used to run these simulations?

**[00:17:33 – 00:17:40]** So all the soft body particle and clock simulation is run on GPU.

**[00:17:40 – 00:17:43]** The reason for that is because compared with CPU, GPU has a

**[00:17:43 – 00:17:46]** lot more computation power, right?

**[00:17:46 – 00:17:49]** So it's actually

**[00:17:49 – 00:17:52]** For example, one of the jerry in there, you're talking about maybe

**[00:17:53 – 00:17:55]** a few thousand elements inside.

**[00:17:55 – 00:17:59]** So if you want to run all this simulation in real time,

**[00:18:00 – 00:18:04]** I think the only choice we have at the moment is in GPU.

**[00:18:04 – 00:18:07]** Right, but I guess as far as specific hardware, I think we're

**[00:18:07 – 00:18:10]** running these on the A6000, right?

**[00:18:10 – 00:18:13]** You can run on A6000 and 3080, 3090, or it's doable.

**[00:18:15 – 00:18:17]** Yeah, all right, another question.

**[00:18:17 – 00:18:21]** How long before physics supports simulating interaction with soil?

**[00:18:21 – 00:18:22]** It's highly important for agriculture and

**[00:18:23 – 00:18:26]** construction industries.

**[00:18:27 – 00:18:28]** Kia, do you want to pick this one?

**[00:18:28 – 00:18:32]** Because I do think we have some capacity to do it and

**[00:18:33 – 00:18:37]** maybe Kia you can talk about that.

**[00:18:37 – 00:18:40]** Yeah, so we're coming at this from a few different angles.

**[00:18:40 – 00:18:43]** And from the pure interaction of particles for soil simulation,

**[00:18:44 – 00:18:49]** we've got a couple of experimental implementations, one using

**[00:18:49 – 00:18:54]** like an XPBD derivation that's able to handle compaction and

**[00:18:54 – 00:18:57]** adhesion for soil, so you can kind of pack it together, so it's not

**[00:18:57 – 00:19:00]** just purely free granular material.

**[00:19:00 – 00:19:03]** And we've also got an experimental implementation of MPM, which

**[00:19:03 – 00:19:05]** is material point method that can do something similar as

**[00:19:05 – 00:19:08]** well, so we're working at least from two different angles.

**[00:19:08 – 00:19:14]** NPM is obviously more expensive than XPBD because, well, for a

**[00:19:14 – 00:19:18]** start, the way that it's simulated, you either need to do an implicit

**[00:19:18 – 00:19:21]** solve on like quite a complex linear system or you end up having

**[00:19:21 – 00:19:24]** to take quite small explicit steps.

**[00:19:24 – 00:19:28]** Whereas XPBD is kind of implicitly stable derivation, so you

**[00:19:28 – 00:19:30]** can do quite large steps, so it's potentially quite

**[00:19:30 – 00:19:31]** a bit faster at the moment.

**[00:19:32 – 00:19:34]** But with either of these, to simulate something like

**[00:19:34 – 00:19:38]** soil, I mean, you could simulate like the topsoil.

**[00:19:38 – 00:19:39]** For example, but but then you've got a large amount

**[00:19:40 – 00:19:42]** underneath and I think you need to bring in some kind of

**[00:19:42 – 00:19:46]** level of hybrid simulation, which is something that we are at least

**[00:19:47 – 00:19:49]** internally discussing, but not necessarily if we haven't actually

**[00:19:49 – 00:19:51]** started to work on that yet.

**[00:19:51 – 00:19:53]** But that's quite an important factor, because obviously if you

**[00:19:53 – 00:19:56]** want to kind of simulate something where you're digging through things

**[00:19:56 – 00:19:59]** and like excavating lumps of soil, I mean you don't really want to

**[00:19:59 – 00:20:02]** be simulating just a small patch, you want to be able to simulate a

**[00:20:02 – 00:20:04]** relatively large area of the world.

**[00:20:04 – 00:20:07]** Which means, yeah, the ability to have this hybrid simulation where

**[00:20:07 – 00:20:11]** you can kind of dig out sections and have them on demand turn into

**[00:20:11 – 00:20:15]** particles or something that you can kind of simulate in high fidelity.

**[00:20:15 – 00:20:17]** And then when you tip it over, it becomes something that's

**[00:20:17 – 00:20:21]** kind of more of a structured kind of mesh that represents the pile

**[00:20:21 – 00:20:22]** once it's kind of come to rest.

**[00:20:22 – 00:20:26]** So we need to put at least some work into researching

**[00:20:26 – 00:20:29]** into that with the team to find out how we can do it.

**[00:20:29 – 00:20:33]** But another way also to basically soil simulation, you're talking

**[00:20:33 – 00:20:36]** about scalability in there, right?

**[00:20:36 – 00:20:39]** Another approach is we can also experiment with multi-GPU.

**[00:20:39 – 00:20:45]** So basically dissect the whole world, not exactly, sorry,

**[00:20:45 – 00:20:48]** not exactly the whole world, maybe all the soil ground

**[00:20:48 – 00:20:50]** you want to simulate, right?

**[00:20:50 – 00:20:52]** And dissect it into different regions.

**[00:20:52 – 00:20:56]** And then we need to deal with all this boundary condition.

**[00:20:56 – 00:21:01]** and dissect the simulation into different nodes and then the data

**[00:21:01 – 00:21:05]** transform communicate between the node itself to scale up like that.

**[00:21:05 – 00:21:11]** So it's got various approach to do it but at the moment it's all

**[00:21:11 – 00:21:17]** in a research stage and then we of course we need to identify the pros

**[00:21:17 – 00:21:19]** and cons of different approach.

**[00:21:19 – 00:21:22]** But yes we have all this.

**[00:21:22 – 00:21:25]** We know soil simulation is important.

**[00:21:25 – 00:21:29]** We're trying to address that in our site as well.

**[00:21:29 – 00:21:34]** Right, so there's a particular question for the Jelly demo.

**[00:21:34 – 00:21:36]** So is the robot arm being driven from within Omni or is it showing

**[00:21:37 – 00:21:38]** a lifelink from an external app?

**[00:21:38 – 00:21:42]** I mean, for the demo, this is being driven inside Omniverse,

**[00:21:42 – 00:21:49]** but I guess we are able to connect it to an external driver, right?

**[00:21:49 – 00:21:51]** Yeah, I mean, if you're looking at something like Isaac Sim,

**[00:21:51 – 00:21:54]** they've got the ROS bridges, and from that they can bring in,

**[00:21:54 – 00:21:59]** you know, from whatever interfaces with ROS, I mean, I don't

**[00:21:59 – 00:22:02]** work on that project directly, but as I understand it, that project

**[00:22:02 – 00:22:05]** is capable of pretty much emulating the interface for a robot,

**[00:22:06 – 00:22:09]** so from a software perspective, you know, it would be possible to

**[00:22:09 – 00:22:14]** bring in from any external project, any external system to drive it.

**[00:22:14 – 00:22:17]** Yeah, so for the articulation system, so the robot arm is

**[00:22:17 – 00:22:19]** actually constructed by the articulation system, right?

**[00:22:19 – 00:22:24]** So for Vset, we actually lead the joint position.

**[00:22:24 – 00:22:27]** Basically, the outside, either you write the logic

**[00:22:28 – 00:22:31]** in script or through the external control system, right?

**[00:22:31 – 00:22:35]** You provide us a joint position and then we update

**[00:22:35 – 00:22:37]** everything for you inside.

**[00:22:37 – 00:22:40]** Right, there's a very general question.

**[00:22:40 – 00:22:43]** If this simulation can be used for testing real car designs, if yes,

**[00:22:43 – 00:22:47]** how would it compare with existing software such as from ANSYS?

**[00:22:47 – 00:22:53]** I'm not sure we have any specialists on this right now.

**[00:22:53 – 00:22:58]** I can say that we've got some

**[00:22:58 – 00:23:00]** validation that's been done against ANSYS for a certain

**[00:23:00 – 00:23:01]** subset of things.

**[00:23:01 – 00:23:03]** So for example,

**[00:23:03 – 00:23:07]** We don't have like the rigid body model and the kind of

**[00:23:07 – 00:23:10]** drive model validated, obviously.

**[00:23:10 – 00:23:12]** And that would be there would be quite a lot of work to bring

**[00:23:12 – 00:23:17]** that up to something where it was kind of close to a realistic model.

**[00:23:17 – 00:23:19]** If we went down the route of doing using something like Featherstone

**[00:23:19 – 00:23:21]** articulations, which is also perfectly possible to implement

**[00:23:21 – 00:23:27]** this, then at least you've got with that, you've got some level,

**[00:23:27 – 00:23:31]** a slightly higher level of accuracy in terms of the way that it.

**[00:23:31 – 00:23:34]** um the way that the the forces propagate in the way that

**[00:23:34 – 00:23:38]** the mass um of the system is simulated which actually matches

**[00:23:38 – 00:23:40]** some analytical models but again i don't think that would necessarily

**[00:23:40 – 00:23:44]** match or come close to matching a real kind of cars kind of

**[00:23:44 – 00:23:49]** um structure and how it works um i mean this is basically replicating

**[00:23:49 – 00:23:54]** how the the lego model works which is a very simplified model

**[00:23:54 – 00:23:57]** If we're talking about things like FEM then for the FEM

**[00:23:57 – 00:24:03]** we do have some validations being done against ANSYS comparing our

**[00:24:03 – 00:24:08]** co-rotated model with their kind of the models that they use which are

**[00:24:08 – 00:24:12]** using really complex direct solvers and the analysis and comparison

**[00:24:12 – 00:24:14]** between those is actually quite favourable that we were able

**[00:24:14 – 00:24:19]** to do something that's within some tolerance and is able to calculate

**[00:24:19 – 00:24:21]** the same levels of deformation the same levels of stress.

**[00:24:22 – 00:24:26]** and strain in real time rather than being non-real time for

**[00:24:26 – 00:24:27]** their approaches.

**[00:24:27 – 00:24:31]** So, but obviously that's been tested to a limited degree

**[00:24:31 – 00:24:34]** with relatively small deformations, obviously as the deformations

**[00:24:34 – 00:24:38]** get larger and the systems become more complicated and more deformed

**[00:24:38 – 00:24:41]** and more stressed, we'd need to

**[00:24:41 – 00:24:45]** Do more work on that to kind of validate against those, but yeah.

**[00:24:45 – 00:24:49]** Yeah, but we also now have a breakthrough for our support

**[00:24:49 – 00:24:51]** implementation as well, right?

**[00:24:51 – 00:24:53]** So we not only support co-rotation model, we

**[00:24:53 – 00:24:55]** actually support hybrid model.

**[00:24:55 – 00:24:58]** So you have a little hook here, non-linear,

**[00:24:58 – 00:24:59]** so we're inside as well.

**[00:24:59 – 00:25:04]** So we make sure the Warren approximation is correct.

**[00:25:04 – 00:25:08]** So all this is coming in 103 release.

**[00:25:09 – 00:25:13]** Okay, there's also another generic question as far as

**[00:25:13 – 00:25:16]** how these kind of physics simulations will benefit from multi

**[00:25:16 – 00:25:20]** GPU or will dedicated GPU only for physics give a decent speed

**[00:25:20 – 00:25:23]** up from a performance standpoint.

**[00:25:25 – 00:25:27]** Who wants to jump in for this?

**[00:25:27 – 00:25:29]** I can answer, or do you want to, Michelle?

**[00:25:29 – 00:25:31]** Don't mind. You go for it.

**[00:25:31 – 00:25:33]** Okay.

**[00:25:33 – 00:25:35]** So from the perspective of a dedicated GPU, then yeah,

**[00:25:35 – 00:25:38]** you'll definitely get improved performance with using a dedicated

**[00:25:38 – 00:25:42]** GPU for PhysX when you're working with Omniverse simply because

**[00:25:42 – 00:25:48]** then it's not competing for compute resources with the renderer.

**[00:25:48 – 00:25:50]** When we're talking about multi-GPU, it depends what

**[00:25:50 – 00:25:51]** you're trying to simulate.

**[00:25:51 – 00:25:55]** So there's scope, for example, with things like

**[00:25:55 – 00:25:58]** The separate environment kind of simulations that we're using

**[00:25:58 – 00:26:03]** in something like iZoom for RL training where we could scale that

**[00:26:03 – 00:26:09]** up already basically with multiple GPUs using techniques like Horovod.

**[00:26:09 – 00:26:13]** If we're talking about taking one singular simulation and scaling

**[00:26:13 – 00:26:17]** up to multi-GPU, I think that is something that there's definitely

**[00:26:17 – 00:26:21]** scope to do and I think it would be very interesting to work on.

**[00:26:22 – 00:26:25]** We don't have that done yet inside PhysX so I couldn't

**[00:26:25 – 00:26:28]** make a statement about how far along we could get with

**[00:26:28 – 00:26:30]** it but I think there's at least scope for some things

**[00:26:30 – 00:26:33]** like particle simulation to really get that scaled up, especially

**[00:26:33 – 00:26:38]** when you're looking at approaches like NPM or FLIP where they're

**[00:26:38 – 00:26:41]** using, for example, grid-based approaches and they're solving

**[00:26:41 – 00:26:47]** pressure and density equations and in those cases, there's very

**[00:26:47 – 00:26:49]** well-defined kind of separations.

**[00:26:50 – 00:26:52]** Where you can dissolve around the boundaries.

**[00:26:52 – 00:26:55]** I'd love to work on this for rigid bodies and various

**[00:26:55 – 00:26:55]** other things as well.

**[00:26:55 – 00:26:58]** I think it's a really interesting, really challenging problem.

**[00:26:59 – 00:27:00]** But yeah, we haven't done it yet.

**[00:27:00 – 00:27:04]** Yeah, but for the RRO case, right, because it's an individual

**[00:27:04 – 00:27:07]** environment that is really well defined, you can already

**[00:27:07 – 00:27:09]** do multi-GPU support.

**[00:27:09 – 00:27:11]** So you have the capacity to do that.

**[00:27:11 – 00:27:15]** So another thing Kia talking about, it's the huge simulation.

**[00:27:15 – 00:27:16]** You have median interaction.

**[00:27:16 – 00:27:19]** How do we separate the work into different GPU?

**[00:27:19 – 00:27:21]** So that is the challenge in that.

**[00:27:21 – 00:27:23]** I think Andrew is the expert on that as well.

**[00:27:23 – 00:27:27]** Maybe Andrew can give us some insight on that.

**[00:27:30 – 00:27:34]** Um, yeah, I mean, from the, the, the flow side, um, I

**[00:27:34 – 00:27:40]** also have, uh, sort of the, the very first level of multi GPU at

**[00:27:40 – 00:27:43]** the moment, which is not to scale the simulation, but the rendering.

**[00:27:43 – 00:27:46]** And a lot of that comes in with like the, the fire when we want to

**[00:27:46 – 00:27:51]** do path tracing and the, and have the, the fire illuminate the scene.

**[00:27:51 – 00:27:53]** And, and so then the sort of

**[00:27:53 – 00:27:56]** Compute load shifts more to the rendering side than

**[00:27:56 – 00:27:57]** the simulation side.

**[00:27:57 – 00:28:02]** So Flow has the ability to move the simulation on a whole

**[00:28:02 – 00:28:06]** series of GPUs and lock step local to the GPU to save memory

**[00:28:06 – 00:28:13]** transfer and then render from that.

**[00:28:13 – 00:28:21]** It's also future work for me to actually make the simulation

**[00:28:21 – 00:28:22]** distribute across the nodes.

**[00:28:22 – 00:28:25]** A lot of that

**[00:28:25 – 00:28:28]** It's also going to depend on the sort of hardware we run on.

**[00:28:28 – 00:28:31]** If we're connected by NVLink, we've got a lot more options

**[00:28:31 – 00:28:34]** to move data around.

**[00:28:34 – 00:28:38]** If we have to assume PCIe or something potentially even

**[00:28:38 – 00:28:41]** slower like a network link, then it's going to get more

**[00:28:42 – 00:28:45]** complicated to scale things.

**[00:28:45 – 00:28:49]** So yeah, we're definitely looking at that.

**[00:28:51 – 00:28:55]** Mikel, should I do another video or do you have a question?

**[00:28:56 – 00:29:01]** Yeah, there's the last question connected to the jelly cubes.

**[00:29:01 – 00:29:04]** So Eric is asking if I understood it well, all jerry cubes are

**[00:29:04 – 00:29:08]** simulated using Tetra FEM, but what technique is used

**[00:29:08 – 00:29:10]** for the collision with the grasper?

**[00:29:10 – 00:29:12]** Are you using penalty constraints?

**[00:29:12 – 00:29:13]** Is it also computed on GPU?

**[00:29:13 – 00:29:17]** So I guess that's also on here.

**[00:29:18 – 00:29:19]** It could be me or Michelle.

**[00:29:20 – 00:29:23]** Okay I can do it.

**[00:29:23 – 00:29:27]** Yeah so it's using tetrahedral mesh collisions so we modeled

**[00:29:27 – 00:29:30]** the the FEM as a set of tetrahedra and we're able

**[00:29:30 – 00:29:32]** to do collision against that.

**[00:29:32 – 00:29:35]** Those contacts are then converted into varicentric coordinates that

**[00:29:35 – 00:29:37]** are used through the simulation so when we're simulating them

**[00:29:37 – 00:29:40]** we're actually kind of stepping simulation and moving things

**[00:29:40 – 00:29:42]** around to kind of push things out.

**[00:29:42 – 00:29:45]** The constraints are not necessarily penalty it's not like a penalty

**[00:29:45 – 00:29:47]** force it's not fixed force but instead it's using Lagrange

**[00:29:47 – 00:29:50]** multipliers so again it's calculating the effective mass

**[00:29:50 – 00:29:56]** of the contacts at those particular points and we can use this

**[00:29:56 – 00:30:00]** we can use a technique where we can look at by exploiting the continuum

**[00:30:00 – 00:30:03]** dynamics and the property of it being tessellation independent that

**[00:30:03 – 00:30:08]** we can look at at the the model of the simulation at different kind of

**[00:30:08 – 00:30:12]** Resolutions as well by using a separation between the simulation

**[00:30:12 – 00:30:12]** mesh and the collision mesh.

**[00:30:12 – 00:30:14]** So we've shown this on some videos before.

**[00:30:14 – 00:30:19]** So one of the techniques that we use in PhysX is we can use the same

**[00:30:19 – 00:30:23]** mesh for collision and simulation, but we can also use a different

**[00:30:23 – 00:30:28]** mesh where we use something like, say, a hexahedral cage

**[00:30:28 – 00:30:31]** at variable resolution to represent the actual simulation for the

**[00:30:32 – 00:30:35]** continuum dynamics for the volume preservation And also for the

**[00:30:35 – 00:30:39]** solving the contacts and solving any attachments and constraints.

**[00:30:39 – 00:30:41]** And then we embed the collision mesh in there.

**[00:30:41 – 00:30:44]** So we generate the contacts at specific points in barycentric

**[00:30:44 – 00:30:48]** space inside this underlying simulation cage, and then

**[00:30:48 – 00:30:49]** you're able to move that around.

**[00:30:49 – 00:30:51]** And that has a lot of nice properties that you get this

**[00:30:51 – 00:30:55]** kind of regularized structure, which allows it to converge

**[00:30:55 – 00:30:59]** a lot better and work a lot better in a simulator.

**[00:30:59 – 00:31:03]** Yeah, I think the main point here is a penalty force, right?

**[00:31:03 – 00:31:05]** It's a Lagrange multiplier.

**[00:31:05 – 00:31:11]** And of course, we simulate a fission on top of the contact

**[00:31:11 – 00:31:13]** the penalty force as well.

**[00:31:13 – 00:31:17]** So when we leave it up, it's actually an actual fission.

**[00:31:17 – 00:31:20]** Leave up the JDQ instead of create a constraint or something.

**[00:31:20 – 00:31:22]** It's actual fission.

**[00:31:22 – 00:31:24]** Yeah, and the last part of the question, is it computed on GPU?

**[00:31:24 – 00:31:29]** Yeah, the whole simulation is GPU.

**[00:31:29 – 00:31:33]** I guess we can continue into another demo.

**[00:32:23 – 00:32:26]** So in that demo, it's clearly for the clothing.

**[00:32:26 – 00:32:30]** So in the digital human, a demo just now, right?

**[00:32:30 – 00:32:34]** We saw the t-shirt, it's actually a piece of cloth

**[00:32:34 – 00:32:35]** and the scar as well.

**[00:32:35 – 00:32:39]** So you can, in real time, interact with them.

**[00:32:39 – 00:32:44]** So we, in Vset, we have two clothing approach.

**[00:32:44 – 00:32:46]** One is using particle base.

**[00:32:46 – 00:32:50]** That is based on master spring system and the video you saw in

**[00:32:51 – 00:32:56]** there is actually using the master spring system and we have FEM

**[00:32:56 – 00:33:03]** cloth work in progress so that will be coming in our future release.

**[00:33:10 – 00:33:11]** Do we have question on that?

**[00:33:11 – 00:33:14]** If not, I can go for another video.

**[00:33:14 – 00:33:17]** I think we have one coming for Andrew.

**[00:33:17 – 00:33:21]** Maybe our, our dear leader, Mikael can read it out.

**[00:33:21 – 00:33:25]** Right, right. So, uh, question for Andrew, uh,

**[00:33:25 – 00:33:27]** From Kyle.

**[00:33:27 – 00:33:30]** They've had a discussion at work of creating digital twins for wind

**[00:33:30 – 00:33:34]** tunnel tests, generate a trail file that can rerun a given wind tunnel

**[00:33:34 – 00:33:36]** test using the collected data.

**[00:33:36 – 00:33:38]** Can you comment on the feasibility of this?

**[00:33:39 – 00:33:41]** A wind tunnel is basically a big doughnut of internal

**[00:33:41 – 00:33:45]** flow with some wind tunnels reaching supersonic speeds.

**[00:33:45 – 00:33:53]** So you'll see some demos at GTC that will be showing sort of

**[00:33:53 – 00:33:59]** Things similar to wind tunnels like windmill-type use cases.

**[00:33:59 – 00:34:03]** And what's happening today is flow's being used to help visualize

**[00:34:03 – 00:34:08]** that information in CREAT.

**[00:34:09 – 00:34:14]** And basically that's just using the ability to import nano

**[00:34:14 – 00:34:18]** VDBs to flow and render it out.

**[00:34:18 – 00:34:21]** For the actual underlying simulation, since some of

**[00:34:22 – 00:34:27]** these are, they're looking for like CFD quality results,

**[00:34:27 – 00:34:31]** they're using one of our other technologies, I believe called

**[00:34:31 – 00:34:39]** Modulus, that is using DL type techniques to accelerate generating

**[00:34:39 – 00:34:42]** a CFD accurate type result.

**[00:34:43 – 00:34:46]** So, you know, using flow directly to simulate a wind

**[00:34:46 – 00:34:48]** tunnel, We've kind of

**[00:34:48 – 00:34:52]** I've done it before as like a visualization thing, but it's not

**[00:34:52 – 00:34:56]** anything that's been validated to be accurate or anything like that.

**[00:34:56 – 00:35:00]** So the main use case today is definitely more to use flow

**[00:35:00 – 00:35:07]** to visualize a more accurate result so that you can see it and create.

**[00:35:14 – 00:35:18]** Okay, so yeah, we have also a couple more questions as far

**[00:35:18 – 00:35:23]** as digital twins, so this is for...

**[00:35:24 – 00:35:25]** No one in particular.

**[00:35:25 – 00:35:27]** Currently, to do a factory real-time simulation at Digital

**[00:35:27 – 00:35:31]** Twin, you need to use live connect tools such as Siemens

**[00:35:31 – 00:35:34]** to build the actual simulation and use Omniverse to display it, along

**[00:35:35 – 00:35:37]** with other live connect simulations to create an overall view.

**[00:35:37 – 00:35:39]** However, the different simulations being live

**[00:35:39 – 00:35:41]** cannot interact with each other.

**[00:35:41 – 00:35:44]** So is there a strategy to start to move more of these capabilities

**[00:35:44 – 00:35:47]** directly into Omniverse with extensions so you can start

**[00:35:47 – 00:35:50]** to build out Digital Twins directly in Omniverse without depending

**[00:35:50 – 00:35:53]** on the other external tools?

**[00:35:55 – 00:35:58]** Yeah, I guess I can partially answer to that.

**[00:35:58 – 00:36:03]** So, yeah, I mean, Isaac Sim is trying to add lots of these tools.

**[00:36:03 – 00:36:11]** And on top of that, we are trying to build physics generic enough to

**[00:36:11 – 00:36:15]** be usable by also other simulation engines, not just physics.

**[00:36:15 – 00:36:21]** So definitely it should be possible to have multiple

**[00:36:21 – 00:36:26]** Tools integrated into Omniverse being Omniverse extensions

**[00:36:26 – 00:36:31]** and simulate what you need, but currently we have the OmniPhysics

**[00:36:31 – 00:36:36]** extension only taking care of the simulation and there are already

**[00:36:36 – 00:36:42]** customers interested in creating own extensions for simulation

**[00:36:42 – 00:36:48]** using different engines, but that's currently not in this release.

**[00:36:48 – 00:36:53]** And probably depends on the customer priorities, but it

**[00:36:53 – 00:36:56]** will come, it's definitely the ultimate goal to connect

**[00:36:56 – 00:37:01]** all the technologies, but so far there's only physics

**[00:37:01 – 00:37:04]** at this very moment, but

**[00:37:04 – 00:37:08]** The pipeline, the USD pipeline has been split into generic part

**[00:37:09 – 00:37:14]** that is USD physics part which is right now part of Pixar's standard

**[00:37:14 – 00:37:21]** and that's something we want to retain and it will be even more

**[00:37:21 – 00:37:28]** generic like the UI will work only with these USD physics core parts.

**[00:37:32 – 00:37:34]** Hey, thanks, Alex.

**[00:37:34 – 00:37:38]** There's a question that's, I guess, partially also for Andrew.

**[00:37:38 – 00:37:43]** Will the cloth be affected by things such as wind?

**[00:37:43 – 00:37:45]** Yeah, I think here it was mentioning that we might have

**[00:37:45 – 00:37:50]** a demo video for that, but I'm not sure we have it uploaded, right?

**[00:37:50 – 00:37:52]** Yeah, we're trying to source it right now to

**[00:37:52 – 00:37:53]** show that it does work.

**[00:37:53 – 00:37:55]** Yes, this is something that we can do.

**[00:37:55 – 00:38:00]** So cloth and soft bodies and things like that can be affected

**[00:38:00 – 00:38:01]** by a wind source.

**[00:38:01 – 00:38:06]** However, at the moment, it's limited to just like kind

**[00:38:06 – 00:38:09]** of localized wind vectors for that particular model

**[00:38:09 – 00:38:10]** that you're simulating.

**[00:38:10 – 00:38:14]** However, yeah, going forwards, we're hoping that we can leverage

**[00:38:14 – 00:38:18]** some of Andrew's amazing work with either VDBs or with flow

**[00:38:18 – 00:38:21]** simulation natively so that we can extract wind forces

**[00:38:22 – 00:38:28]** out of that and do something far more interesting and dynamic.

**[00:38:29 – 00:38:32]** Right, I think we found the video, but not sure if we

**[00:38:32 – 00:38:34]** have it already uploaded.

**[00:38:34 – 00:38:38]** We need to upload it before we can play it because we

**[00:38:38 – 00:38:39]** cannot share the screen in here.

**[00:38:39 – 00:38:42]** Okay, so let's see if there's, if there are

**[00:38:42 – 00:38:45]** more questions in the meantime.

**[00:38:46 – 00:38:48]** There's a question, are there more options for CFD coming

**[00:38:48 – 00:38:52]** within or outside of Omniverse?

**[00:38:53 – 00:39:00]** I think NVIDIA Modulus is probably the main sort of CFD accurate

**[00:39:00 – 00:39:01]** solution that we're working on.

**[00:39:01 – 00:39:08]** A lot of our focus on the, and for the coming from outside, part

**[00:39:08 – 00:39:11]** of what we've been trying to do is sort of consolidate the volume

**[00:39:11 – 00:39:19]** format standard with Omniverse, so OpenVDB and NanoVDB, which

**[00:39:19 – 00:39:21]** is sort of an extension of OpenVDB.

**[00:39:21 – 00:39:26]** We're supporting as broadly as possible with the hope

**[00:39:26 – 00:39:30]** that when people have these external simulations and they want

**[00:39:30 – 00:39:33]** to visualize an omniverse, it's just a matter of exporting some

**[00:39:33 – 00:39:39]** VDBs, pulling them in to create and being able to play them back.

**[00:39:42 – 00:39:46]** Okay, I think we've run out of questions at this point.

**[00:39:46 – 00:39:51]** There's actually one which was very general.

**[00:39:51 – 00:39:54]** So the question was, are the simulations made with code,

**[00:39:54 – 00:39:57]** like coding, the laws of physics, or are they made with AI,

**[00:39:57 – 00:39:58]** like AI-driven physics?

**[00:39:58 – 00:40:03]** And if not, would it be much more computationally expensive to do it?

**[00:40:03 – 00:40:08]** I think we have our, is it a simulation question here.

**[00:40:09 – 00:40:13]** So I can say that there is open research to look to see if we can

**[00:40:13 – 00:40:19]** replace elements of fixed-function simulation with learned models.

**[00:40:19 – 00:40:22]** I think in some cases, for a lot of things that we're

**[00:40:22 – 00:40:26]** showing here, the possibility of it being faster with AI

**[00:40:26 – 00:40:33]** is probably unlikely just because of the kind of level of scale that

**[00:40:33 – 00:40:36]** you'd have to simulate, but there is the possibility that AI could

**[00:40:36 – 00:40:39]** be able to simulate phenomena that aren't able to be expressed as

**[00:40:39 – 00:40:45]** easily in our kind of simulations that we're simulating.

**[00:40:45 – 00:40:53]** So, for example, there's a lot of complexity that classic

**[00:40:53 – 00:40:56]** physics simulations

**[00:40:58 – 00:41:01]** Brush under the carpet, let's say, in the scope for trying

**[00:41:01 – 00:41:06]** to get a nice closed-form model that AI could potentially fill in

**[00:41:06 – 00:41:12]** the blanks for by using real-world observations to train a model.

**[00:41:12 – 00:41:15]** And whether that actually fully replaces a simulation or just

**[00:41:15 – 00:41:18]** augments it and adds additional layers to it is hard to say.

**[00:41:18 – 00:41:21]** There's also the possibility that you may have some problems

**[00:41:21 – 00:41:23]** that AI can simply win at.

**[00:41:23 – 00:41:26]** because it's able to do things that the simulation isn't.

**[00:41:26 – 00:41:30]** So for example, other cases might be simulations where

**[00:41:30 – 00:41:33]** we are required to simulate very small time steps to be

**[00:41:33 – 00:41:39]** able to simulate them stably, but AI is not restricted by that.

**[00:41:39 – 00:41:41]** Basically, AI would allow you to take larger steps by being able

**[00:41:41 – 00:41:46]** to figure out what's actually going on without necessarily having to do

**[00:41:46 – 00:41:51]** the kind of, you know, numerically unstable integration of huge.

**[00:41:51 – 00:41:56]** um stiff springs or large forces or various things that that force

**[00:41:56 – 00:42:00]** us to take small time steps so there's there is scope definitely

**[00:42:00 – 00:42:06]** Well, we have some research going on for the DL Crocs as well, right?

**[00:42:06 – 00:42:11]** So the whole idea of that is they take a low-resolution model

**[00:42:11 – 00:42:16]** and then up-sample it, so they use DL to trade the model, right?

**[00:42:16 – 00:42:20]** And then in real-time, they replace the low-definition model and

**[00:42:21 – 00:42:23]** use the up-sampled model in there.

**[00:42:23 – 00:42:27]** So that is one of the results we are able to achieve right now.

**[00:42:27 – 00:42:31]** The other, using AI as well.

**[00:42:31 – 00:42:36]** Honestly, if for simulation, let's just say just a fluid dynamite,

**[00:42:36 – 00:42:40]** because the dynamite when you solve in that part is so simple.

**[00:42:40 – 00:42:46]** So DL, I think it has a difficulty to try to replace a simulation

**[00:42:46 – 00:42:48]** in terms of performance.

**[00:42:48 – 00:42:52]** but if you're talking about some also inside right so for example

**[00:42:52 – 00:42:57]** using dl to figure out uh now i wear the clothes right so which

**[00:42:57 – 00:43:00]** configuration this goes on my body it will be let's go to the pinch

**[00:43:00 – 00:43:05]** condition and let let um let's go to the south health condition

**[00:43:05 – 00:43:06]** and time coding condition.

**[00:43:06 – 00:43:09]** So the DL is actually can help us.

**[00:43:09 – 00:43:13]** to place a different constraint into the model and then make sure

**[00:43:13 – 00:43:17]** it's intact with the rigid body itself so limit the movement in

**[00:43:17 – 00:43:24]** there so um in this sense i think dl is extremely useful there's

**[00:43:24 – 00:43:27]** also the side of looking at things like npm where for example we

**[00:43:27 – 00:43:31]** might need to simulate it something like you know 5 000 or 10 000 hertz

**[00:43:31 – 00:43:34]** to get like a stable simulation of something where we go able

**[00:43:34 – 00:43:39]** to model like rigidity of things that you want so very rigid models

**[00:43:40 – 00:43:43]** And it's quite possible that either using some kind of

**[00:43:43 – 00:43:46]** hybrid approach or using some kind of AI, we could move that forward,

**[00:43:46 – 00:43:49]** maybe an order or two orders of magnitude, larger time steps,

**[00:43:49 – 00:43:53]** so that it approaches something that you can do in real time.

**[00:43:53 – 00:43:55]** So that's definitely a possibility as well.

**[00:43:55 – 00:44:00]** And that's one case where the AI may be on a pure frame-for-frame

**[00:44:00 – 00:44:03]** basis, because NPM itself is a small amount of code

**[00:44:03 – 00:44:05]** and it's relatively simple.

**[00:44:05 – 00:44:09]** But where the complexity lies in is just on how

**[00:44:09 – 00:44:10]** how frequently you have to run it.

**[00:44:10 – 00:44:13]** So with that, yeah, with AI, potentially you could run

**[00:44:13 – 00:44:17]** it at much larger time steps, and it would be an order of magnitude

**[00:44:17 – 00:44:18]** faster if we can get that working.

**[00:44:18 – 00:44:23]** So there's definitely scope as well for simulation side.

**[00:44:25 – 00:44:28]** All right, I think we've uploaded the wind video, so we can

**[00:44:28 – 00:44:30]** try playing that.

**[00:44:30 – 00:44:33]** So Michelle, can you try playing the

**[00:44:34 – 00:44:39]** It's actually called blocky, the file, or I can play it.

**[00:44:39 – 00:44:40]** If you know where it is, go for it.

**[00:44:40 – 00:44:42]** Yeah, yeah, okay. Do you know where is that, go?

**[00:44:42 – 00:44:46]** Yeah, all right.

**[00:45:04 – 00:45:07]** quite a problem that shows that wind works.

**[00:45:07 – 00:45:10]** But not just so the wind work, right?

**[00:45:10 – 00:45:15]** It's also showing the attachment with the clocks attached to

**[00:45:15 – 00:45:17]** the chair, right?

**[00:45:17 – 00:45:19]** So we have the auto-attachment capacity.

**[00:45:19 – 00:45:22]** So let you to auto-attach a different

**[00:45:22 – 00:45:24]** constraint into rigid body.

**[00:45:24 – 00:45:27]** Not only that, not only rigid body, we can attach to any

**[00:45:28 – 00:45:32]** of the feature set support, such as soft body, right, and

**[00:45:32 – 00:45:36]** then FEM cloud as well, or particle system into particle system.

**[00:45:36 – 00:45:38]** So it's got lots of capacity.

**[00:45:39 – 00:45:43]** All these features is coming to kids as well.

**[00:45:44 – 00:45:51]** OK, we've got a question about AI, DL, and ML, which is coming up.

**[00:45:51 – 00:45:52]** Cheers.

**[00:45:52 – 00:45:56]** OK, so the question is, how reliable is AI, DL, ML for

**[00:45:56 – 00:45:59]** nonlinear system simulations?

**[00:45:59 – 00:46:02]** Can it be real time specifically for control of six degree

**[00:46:02 – 00:46:05]** of freedom systems?

**[00:46:07 – 00:46:12]** anyone to jump in and have a guess i think i don't want to guess

**[00:46:12 – 00:46:18]** maybe you can i think that the first step is is we need to do some

**[00:46:18 – 00:46:21]** work on actually trying to do these things so if we're talking about

**[00:46:21 – 00:46:26]** pure control um for non-linear systems um we would need to first

**[00:46:26 – 00:46:29]** build these nonlinear systems build the ai to control them and then

**[00:46:29 – 00:46:33]** we need to measure it and somehow validate it and that's like several

**[00:46:33 – 00:46:36]** steps beyond where we currently are to be able to answer that

**[00:46:36 – 00:46:39]** um and then the question as well is is how we would do

**[00:46:39 – 00:46:42]** this and and if there are other alternatives as well so for

**[00:46:42 – 00:46:47]** example um if the problem itself can be defined in a way that's

**[00:46:47 – 00:46:49]** differentiable then potentially you could calculate the derivatives

**[00:46:49 – 00:46:53]** or the gradients of the function and then you could potentially

**[00:46:53 – 00:46:58]** find solutions without necessarily relying on ai dl or ml but

**[00:46:58 – 00:47:00]** instead just gradient the sense.

**[00:47:00 – 00:47:02]** But again, it really depends on the problem.

**[00:47:02 – 00:47:06]** I mean, with 6DOF systems, we're talking about rotation

**[00:47:06 – 00:47:09]** and translation and whether there are closed functions

**[00:47:09 – 00:47:12]** and closed form ways of doing it.

**[00:47:12 – 00:47:18]** So yeah, unfortunately, not enough information to really answer that.

**[00:47:19 – 00:47:22]** And that is the best we can do.

**[00:47:22 – 00:47:26]** Sorry. No, no, this is another question.

**[00:47:26 – 00:47:29]** So I guess for Andrew, how is the wind represented in

**[00:47:29 – 00:47:31]** the wind cloth interaction?

**[00:47:31 – 00:47:34]** Particles?

**[00:47:37 – 00:47:41]** I mean, from the implementation for that cloth, it's

**[00:47:41 – 00:47:43]** airspeed, basically.

**[00:47:43 – 00:47:46]** So it's airspeed at the triangle.

**[00:47:46 – 00:47:52]** And then it uses a fairly simple kind of drag and lift model based

**[00:47:52 – 00:47:56]** on the orientation and area of the triangles versus the velocity

**[00:47:56 – 00:48:00]** and the velocity of the particles versus the velocity of the wind.

**[00:48:00 – 00:48:02]** And at the moment, yeah, like I say, it's one.

**[00:48:02 – 00:48:06]** It's one simple value, basically, so it's a very simplistic approach.

**[00:48:06 – 00:48:08]** What we're hoping to do is to be able to sample this

**[00:48:08 – 00:48:11]** from something like a VDB from Flow so that we could actually

**[00:48:11 – 00:48:16]** get kind of eddies and vortices and various other things in there

**[00:48:16 – 00:48:18]** that would allow us to be able to simulate that and then potentially

**[00:48:18 – 00:48:22]** as well, things like the cloth itself could then be an occluder.

**[00:48:22 – 00:48:26]** That influences the wind inside flow, if it's simulated, or

**[00:48:26 – 00:48:30]** if it's just a fixed function that's maybe been pre-recorded.

**[00:48:31 – 00:48:33]** Any occlusions from rigid bodies or the environment would

**[00:48:33 – 00:48:35]** at least be factored into that.

**[00:48:36 – 00:48:37]** And then we can get more interesting behavior.

**[00:48:37 – 00:48:39]** More interesting behavior.

**[00:48:39 – 00:48:41]** Yeah, let's clarify this, right, because this one is

**[00:48:42 – 00:48:46]** the wind effect, it's actually a property of our particle system, so

**[00:48:46 – 00:48:51]** the user just need to set a value in there, and then we work out

**[00:48:51 – 00:48:54]** the formula in CypherSat itself.

**[00:48:54 – 00:48:58]** So currently it's not interactable with the flow, but the future

**[00:48:58 – 00:49:03]** plan is we should integrate flow with our particle system,

**[00:49:03 – 00:49:04]** let it influence that.

**[00:49:05 – 00:49:09]** I actually have a video for the defunable on fire.

**[00:49:09 – 00:49:11]** So this is the new tech, right?

**[00:49:11 – 00:49:13]** Let me show it.

**[00:49:56 – 00:50:00]** Cool. So maybe Andrew can give us an explanation about the

**[00:50:00 – 00:50:03]** technique used behind the scene.

**[00:50:04 – 00:50:09]** Yeah, so this is showing a new feature coming out

**[00:50:09 – 00:50:12]** with Flow in 1.0.3.

**[00:50:13 – 00:50:17]** Basically, it's the ability to take in a mesh, in particular the

**[00:50:17 – 00:50:24]** USD mesh description, and rasterize it into the Flow grid in real time.

**[00:50:24 – 00:50:29]** So what's happening there is, PhysX is doing a soft

**[00:50:29 – 00:50:31]** body simulation, it's updating.

**[00:50:31 – 00:50:36]** The USD mesh and then flow is pulling that USD mesh and

**[00:50:36 – 00:50:39]** then rasterizing it into the flow grid in real time and then

**[00:50:39 – 00:50:42]** emitting fire from the surface.

**[00:50:42 – 00:50:45]** And it's even in that video, it's even rasterizing the grid

**[00:50:45 – 00:50:52]** multiple times to try to do sort of a sub-stepping of the motion.

**[00:51:00 – 00:51:05]** I don't really see any questions right now connected to this,

**[00:51:05 – 00:51:08]** but there are still a couple of questions connected to wind.

**[00:51:08 – 00:51:11]** So a follow-up on the wind cloth interaction.

**[00:51:11 – 00:51:14]** Is the wind a uniform vector or is it going

**[00:51:14 – 00:51:18]** to be a non-uniform across space?

**[00:51:18 – 00:51:21]** In that demo, it's a uniform vector, but this is something

**[00:51:21 – 00:51:22]** that we want to replace.

**[00:51:22 – 00:51:25]** Like I say, we want to be able to sample wind as being a

**[00:51:25 – 00:51:30]** local phenomena, and whether that's done with specific emitters that

**[00:51:30 – 00:51:34]** we place in places, and we kind of use some kind of function to

**[00:51:34 – 00:51:37]** blend them, or we use a VDB-style approach where we're using

**[00:51:37 – 00:51:39]** a volumetric wind representation.

**[00:51:39 – 00:51:41]** My preference would definitely be a volumetric wind representation

**[00:51:41 – 00:51:44]** because we've got fantastic implementation from Andrew that

**[00:51:44 – 00:51:48]** we could leverage to do that, so we don't need to reinvent the wheel.

**[00:51:48 – 00:51:52]** And it should be really fast and really scalable.

**[00:51:53 – 00:51:56]** All right, there's another connected to the AI question.

**[00:51:56 – 00:52:00]** So AI will make simulations like, well, actually Peter

**[00:52:00 – 00:52:02]** is asking if he get it right.

**[00:52:02 – 00:52:05]** So if this is true, that AI will make simulations like

**[00:52:05 – 00:52:09]** vibration, shock waves, et cetera, and other chaotic-like

**[00:52:09 – 00:52:11]** systems a lot easier to model.

**[00:52:11 – 00:52:13]** So is this right?

**[00:52:13 – 00:52:15]** It's possible.

**[00:52:15 – 00:52:20]** It depends on whether we can, to get the AI to actually

**[00:52:20 – 00:52:22]** Produce what you want. You need to have sufficient samples

**[00:52:22 – 00:52:25]** that you can actually train with.

**[00:52:25 – 00:52:28]** And then you need to obviously test and validate it against

**[00:52:28 – 00:52:29]** other samples that you've reserved.

**[00:52:29 – 00:52:32]** So if you can simulate this, then we can generate

**[00:52:32 – 00:52:33]** a large number of them.

**[00:52:33 – 00:52:35]** If it's something that we either don't know how to simulate

**[00:52:35 – 00:52:37]** or there aren't simulators out there that know how to

**[00:52:37 – 00:52:42]** do it, or it's exceedingly expensive to try and do, then that

**[00:52:42 – 00:52:44]** might limit how much you can do it.

**[00:52:44 – 00:52:46]** But obviously if you can get real world examples of things that

**[00:52:46 – 00:52:50]** we have no idea how to simulate and an AI can figure it out.

**[00:52:50 – 00:52:53]** Just from looking at those real world examples, if there's a large

**[00:52:53 – 00:52:56]** suite of measurements, which is something that's possibly the case.

**[00:52:56 – 00:53:01]** I mean, there's a huge range of people who are working on research

**[00:53:01 – 00:53:05]** into physics simulations, molecular simulations, various things

**[00:53:05 – 00:53:10]** that are using, that they have real world measurements of things

**[00:53:10 – 00:53:15]** that there aren't actually closed form formulas to kind of represent.

**[00:53:15 – 00:53:19]** Then yeah, AI is something that could potentially

**[00:53:19 – 00:53:23]** Fill in the blanks and add in phenomena that would normally

**[00:53:23 – 00:53:26]** not be simulated.

**[00:53:28 – 00:53:31]** All right, there's a question about differentiability.

**[00:53:31 – 00:53:34]** So there is also a Warp session.

**[00:53:34 – 00:53:38]** And are there any plans of differentiability for physics or

**[00:53:38 – 00:53:42]** combining physics and Warp, where, for example, low-level physics

**[00:53:42 – 00:53:48]** modules would be native, operations disposed at Warp, et cetera?

**[00:53:48 – 00:53:52]** Again, this is something that we're considering and something

**[00:53:52 – 00:53:53]** that we've discussed, but not necessarily something

**[00:53:54 – 00:53:56]** that we have any solid plans on.

**[00:53:56 – 00:53:58]** There are difficulties in

**[00:53:58 – 00:54:03]** Um, in, in differentiability in terms of autodiff, um, just

**[00:54:03 – 00:54:05]** simply because in order to do that, you need to be able to express what

**[00:54:06 – 00:54:11]** it's, um, to be differentiated with respect to, um, at compile time.

**[00:54:11 – 00:54:15]** Um, or you, you need to use up an enormous amount of memory to be

**[00:54:15 – 00:54:17]** able to store enough information to be able to compute the derivatives.

**[00:54:17 – 00:54:22]** Um, and then there's also the question of, is that,

**[00:54:22 – 00:54:25]** is that sufficient or would something else also be good enough,

**[00:54:25 – 00:54:28]** just like finite differencing?

**[00:54:28 – 00:54:30]** Would that get you there? It depends on the number of control

**[00:54:30 – 00:54:31]** variables and various other things.

**[00:54:31 – 00:54:34]** So I think it's something that we're looking at, and

**[00:54:34 – 00:54:36]** we're very interested in Warp.

**[00:54:36 – 00:54:42]** Obviously, Warp is a fully differentiable kind of language

**[00:54:42 – 00:54:45]** that's derived from Python, similar in nature to kind

**[00:54:45 – 00:54:48]** of Tai Chi or JAX.

**[00:54:48 – 00:54:50]** And there's a lot of positives there.

**[00:54:50 – 00:54:53]** And the option of combining elements of VizX and elements

**[00:54:53 – 00:54:56]** of Warp, again, is totally doable.

**[00:54:57 – 00:55:02]** So you can get certain parts that are differentiable and so yeah it's

**[00:55:02 – 00:55:05]** something that we're looking at but it's more research rather than

**[00:55:05 – 00:55:09]** something that we have a product a clear product path for yet.

**[00:55:10 – 00:55:15]** I think log-log just combines with the differentiable ability, right?

**[00:55:15 – 00:55:18]** Just warp itself, because as a language, you can just

**[00:55:18 – 00:55:23]** write the Python code, so like the researcher, for the

**[00:55:23 – 00:55:25]** robotic researcher area, right?

**[00:55:25 – 00:55:27]** So you write the reward function and cost function

**[00:55:27 – 00:55:31]** in Python, and then you use warp to just intercompile

**[00:55:31 – 00:55:35]** the code into CUDA, and then

**[00:55:35 – 00:55:39]** So you can create the actual end-to-end pipeline for us anyway.

**[00:55:39 – 00:55:44]** So you can use the writer cost function, reward function

**[00:55:44 – 00:55:48]** in warp, and then let warp translate into the Python code, and

**[00:55:48 – 00:55:51]** then inject into vset, the backend.

**[00:55:51 – 00:55:55]** So we already done that internally for our molecular dynamic

**[00:55:55 – 00:55:56]** simulation anyway.

**[00:55:56 – 00:56:00]** So the speed we get, it's amazing compared with the PyTorch.

**[00:56:00 – 00:56:04]** I think We have a 5x speedup or something like that.

**[00:56:04 – 00:56:07]** So it's not exclusive, right?

**[00:56:07 – 00:56:09]** You have to use the differentiable ability.

**[00:56:09 – 00:56:12]** That is one way, but the other way, you can see it

**[00:56:12 – 00:56:16]** as a language and then combine with our backend as well.

**[00:56:16 – 00:56:19]** So that will speed up your AI trading anyway.

**[00:56:19 – 00:56:22]** Yeah, and also just serving as something like a bridging

**[00:56:22 – 00:56:22]** language as well.

**[00:56:22 – 00:56:27]** So for example, if we wanted to interface with volumetric data in

**[00:56:27 – 00:56:31]** Andrew's simulation for Flow, and we wanted to get that information

**[00:56:31 – 00:56:33]** applying forces through to

**[00:56:33 – 00:56:37]** Physics cloth or physics soft bodies or rigid bodies.

**[00:56:37 – 00:56:38]** Warp is a fantastic way to do it.

**[00:56:38 – 00:56:44]** We can operate in direct buffer interoperability between CUDA and

**[00:56:44 – 00:56:49]** warp and maybe going through Vulkan or DX or whatever Andrew's using.

**[00:56:49 – 00:56:50]** He can be using CUDA as well.

**[00:56:50 – 00:56:54]** He's got a magic transpiler that works with everything.

**[00:56:54 – 00:56:57]** So it depends which mood he's in.

**[00:56:57 – 00:57:01]** But yeah, so operating on that, then Warp as well is

**[00:57:01 – 00:57:03]** a fantastic kind of bridging language where necessarily

**[00:57:04 – 00:57:08]** PhysX doesn't necessarily need to know the low level details of Flow.

**[00:57:08 – 00:57:11]** Flow doesn't necessarily need to know the low level details

**[00:57:11 – 00:57:14]** of PhysX, but this can act as a bridging mechanism to operate

**[00:57:14 – 00:57:17]** between buffers, between the two.

**[00:57:17 – 00:57:20]** OK, so I think we've run out of time, right?

**[00:57:20 – 00:57:23]** Mikael, do we have any more questions?

**[00:57:23 – 00:57:26]** Yeah, there are still a couple, but I think we've already over.

**[00:57:26 – 00:57:28]** So, um...

**[00:57:28 – 00:57:32]** I could probably answer a couple of those super quick.

**[00:57:32 – 00:57:36]** The question about is Flow using Lattice Boltzmann?

**[00:57:36 – 00:57:42]** The answer is no, it's doing cell-centered semi-Lagrangian.

**[00:57:42 – 00:57:44]** I think we recently talked about Lattice Boltzmann.

**[00:57:45 – 00:57:47]** We're not sure how real time we can make that one yet.

**[00:57:47 – 00:57:51]** So yeah, so Flow definitely is a bias towards things we

**[00:57:51 – 00:57:53]** know can be real time.

**[00:57:53 – 00:57:57]** And then, you know, we'll explore other directions, but we're

**[00:57:57 – 00:58:00]** still trying to stay real time.

**[00:58:00 – 00:58:03]** Chemistry involved in the fire simulation.

**[00:58:03 – 00:58:07]** Not really, it's an incredibly simplified model.

**[00:58:07 – 00:58:10]** And just kind of taking the gaseous fuel and converting

**[00:58:10 – 00:58:19]** it to smoke and then temperature to cause the light emission.

**[00:58:19 – 00:58:23]** We've, you know, we might explore

**[00:58:23 – 00:58:26]** A more detailed model, but that would still be like an opt-in

**[00:58:26 – 00:58:33]** thing because some use cases do not need that level of simulation.

**[00:58:34 – 00:58:37]** Right, there was actually a completely last question, which

**[00:58:37 – 00:58:40]** was, could you combine the wind and the fire and see how wind affects

**[00:58:41 – 00:58:43]** how fire spreads based on the wind?

**[00:58:43 – 00:58:45]** Yeah, you can you can already do that now.

**[00:58:45 – 00:58:47]** You can already have like an emitter that has a velocity

**[00:58:47 – 00:58:50]** and blow on the fire.

**[00:58:50 – 00:58:55]** The one thing that might might be missing is at the moment,

**[00:58:55 – 00:58:58]** there's not really a solid fuel model, right?

**[00:58:58 – 00:59:01]** So it's more like what the fuel you're emitting is currently

**[00:59:01 – 00:59:04]** modeled as just a gas that can

**[00:59:04 – 00:59:06]** Ignite sort of thing.

**[00:59:06 – 00:59:10]** So to really capture like a full forest fire thing, we kind of

**[00:59:10 – 00:59:16]** need to add a solid fuel model that interacts with the flow simulation

**[00:59:16 – 00:59:18]** to really capture all of that.

**[00:59:18 – 00:59:23]** But yes, you can absolutely blow the fire around today.

**[00:59:24 – 00:59:28]** Okay, we are out of time, so hope you guys enjoyed this

**[00:59:28 – 00:59:33]** session and then we see you guys at the next GTC.

**[00:59:34 – 00:59:34]** Bye now.