NVIDIA Omniverse Audio2Face

NVIDIA Audio2Face

Instantly create expressive facial animation from just an audio source using generative AI.

Audio-to-Animation Made Easy With Generative AI

NVIDIA Audio2Face beta is a foundation application for animating 3D characters facial characteristics to match any voice-over track, whether for a game, film, real-time digital assistant, or just for fun. You can use the Universal Scene Description (OpenUSD)-based app for interactive real-time applications or as a traditional facial animation authoring tool. Run the results live or bake them out, it’s up to you.

How It Works

Audio2Face is preloaded with “Digital Mark”— a 3D character model that can be animated with your audio track, so getting started is simple—just select your audio and upload. The audio input is then fed into a pre-trained Deep Neural Network and the output drives the 3D vertices of your character mesh to create the facial animation in real-time. You also have the option to edit various post-processing parameters to edit the performance of your character. The results you see on this page are mostly raw outputs from Audio2Face with little to no post-processing parameters edited.

Omniverse Audio2Face App
 
Audio2Face generates facial animation through audio input.

Audio Input

Use a Recording, or Animate Live

Simply record a voice audio track, input into the app, and see your 3D face come alive. You can even generate facial animations live using a microphone.

Audio2Face will be able to process any language easily. And we’re continually updating with more and more languages.  

Character Transfer

Face-Swap in an Instant

Audio2Face lets you retarget to any 3D human or human-esque face, whether realistic or stylized. This makes swapping characters on the fly—whether human or animal—take just a few clicks.

 
Animate any character face with Audio2Face
 
Use multiple instances to generate facial animation for more than one character

Scale Output

Express Yourself—or Everyone at Once

It’s easy to run multiple instances of Audio2Face with as many characters in a scene as you like - all animated from the same, or different audio tracks. Breathe life and sound into dialogue between a duo, a sing-off between a trio, an in-sync quartet, and beyond. Plus, you can dial up or down the level of facial expression on each face and batch output multiple animation files from multiple audio sources.

Emotion Control

Bring the Drama

Audio2Face gives you the ability to choose and animate your character’s emotions in the wink of an eye. The AI network automatically manipulates the face, eyes, mouth, tongue, and head motion to match your selected emotional range and customized level of intensity, or, automatically infers emotion directly from the audio clip.

 
Add emotions to your animation.
 
Add emotions to your animation.

Data Conversion

Connect and Convert

The latest update to Audio2Face now enables blendshape conversion and also blendweight export options. Plus, the app now supports export-import with Blendshapes for Blender and Epic Games Unreal Engine to generate motion for characters using their respective Omniverse Connectors.

Get Started with Audio2Face in NVIDIA Avatar Cloud Engine

Download available production microservices today or apply for our early access program to explore models in development. ACE delivers the building blocks to deliver interactive digital humans.

Dive into Step-by-Step Tutorials

Get Live Help

Connect with Omniverse experts live to get your questions answered. 

Explore Resources

Learn at your own pace with free getting started material.

Download NVIDIA Audio2Face Authoring Application

Frequently Asked Questions

Become Part of Our Community

Access Tutorials

Take advantage of hundreds of free tutorials, sessions, or our beginner’s training to get started with USD

Become an Omnivore

Join our community! Attend our weekly live streams on Twitch and connect with us on Discord and our forums.

Get Technical Support

Having trouble? Post your questions in the forums for quick guidance from Omniverse experts, or refer to the platform documentation.

Showcase Your Work

Created an Omniverse masterpiece? Submit it to the Omniverse Gallery, where you can get inspired and inspire others.

Connect With Us

Stay up-to-date on the latest NVIDIA Omniverse news.