NVIDIA Omniverse Machinima: A Revolution in Video Game Storytelling

By Richard Kerris on September 01, 2020 | Featured Stories Omniverse Machinima

Machinima is a term used to describe the use of real-time computer graphics engines to create a cinematic production. Typically, these stories are realized by repurposing video game assets to generate animated films for sharing on the web. Creators have been producing these types of online stories using content from a mix of 3D games, assets, and effects since the 1980s.

The medium gained popularity throughout the 90s with the advent of player-made ‘Quake Movies’, constructed from gameplay sequences in id Software’s hit games Doom and Quake. ‘Machinima’ - stemming from ‘machine’ and ‘cinema’ - exploded to mainstream media, seeing it used in every type of production, from television series to advertising, and hit series such as ‘Red vs Blue’. Almost three decades since its inception, Machinima content has had tens of billions of views.

Creating rich and compelling Machinima content has long been a challenge for creators due to the limitations of the game assets and their animation tools. Adding to that is the difficulty in producing narrative driven animations from this approach because of all the combinations necessary for creating believable movements that play smoothly and in context with the overall story. By bringing NVIDIA AI technology and RTX rendering to this process, Machinima creators will now have a rich set of tools with unprecedented capabilities for creating real-time stories and sharing them with the world.

Introducing NVIDIA Omniverse Machinima: A Revolutionary New Storytelling App for Gamers

NVIDIA Omniverse Machinima enables gamers across the world to easily master the art of storytelling by using 3D objects, animated by NVIDIA AI technologies.

During the GeForce RTX 30 Series launch on September 1st, NVIDIA CEO Jensen Huang previewed NVIDIA Omniverse Machinima. The app - built on the NVIDIA Omniverse platform - transforms the traditional Machinima workflow with four key NVIDIA technologies: MDL/materials, Audio to Facial Animation, Physics/VFX, and AI Pose Estimation.

 

Through NVIDIA Omniverse, creators can import assets from supported games or most 3rd party asset libraries into the platform. Creators can then automatically animate characters using an AI-based pose estimator and footage from their webcam. Characters' faces can come to life with only a voice recording using NVIDIA's new Audio2Face technology. Reality is effortlessly recreated with high-fidelity physics like particles and fluids, and physically accurate materials. After mixing, playing, and composing the scenes, path-traced rendering technology is enabled with one-click to final output with the highest fidelity using Omniverse RTX Renderer.

Behind the Scenes: Omniverse Machinima Demo

The Omniverse Machinima demo presented in the GeForce RTX 30 Series launch showcased the app’s ability to unite assets from multiple sources. The preview featured scenes from both TaleWorlds Entertainment’s popular Mount & Blade II: Bannerlord, and third-party assets from Turbosquid. NVIDIA’s inhouse creators combined assets for initial layout, then used NVIDIA PhysX and Flow to add true to life smoke and fire. And NVIDIA’s AI Pose Estimator and Audio2Face were used to drive the new character animation and voice for the final scene. Each of these elements were mixed in the Omniverse Machinima app, and output at stunning quality using the built in RTX path-traced rendering.

 

NVIDIA Omniverse is an open platform built for virtual collaboration and simulation. It provides a universal design tool asset exchange with a viewer based on real-time, photorealistic path-tracing. The engine is designed to be physically accurate - simulating light, physics, materials, and AI. With Omniverse, creators, designers, and engineers can accelerate their workflows with one-click interoperability between leading software tools, and seamless collaboration in an interactive, simulated world.

Gamers interested in NVIDIA Omniverse Machinima can get a behind-the-scenes look at the app and register for beta access, coming in October, by heading here.

Machinima Image courtesy of Mount & Blade II: Bannerlord, TaleWorlds Entertainment c2020 with NVIDIA