At GDC 2024 and GTC 2024 we announced that leading AI developers across a wide range of industries are using NVIDIA digital human technologies to create lifelike avatars for commercial applications and dynamic game characters.
NVIDIA’s digital human suite of technologies includes NVIDIA Avatar Cloud Engine (ACE) for speech and animation, NVIDIA NeMo for language, and NVIDIA RTX for ray-traced rendering, enabling developers to build digital humans capable of AI-powered natural language interactions, making conversations more realistic and dynamic.
Users experience NVIDIA ACE technologies through developer creations made possible by our digital human platform partners. At the shows, developers and platform partners unveiled new digital human technology demos leveraging several NVIDIA AI technologies and NVIDIA ACE microservices. Keep reading to learn how Hippocratic AI, Inworld AI, Perfect World Games, RealityArts Studio and UneeQ are leveraging NVIDIA ACE, NVIDIA NeMo, and NVIDIA RTX technologies in their games, platforms, and applications.
Inworld AI and NVIDIA Collaborate On New Covert Protocol Tech Demo
At GDC 2024, Inworld and NVIDIA collaborated on a new technology demo called Covert Protocol to showcase NVIDIA ACE technologies and the Inworld Engine. In the Covert Protocol tech demo, you act as a private detective who completes objectives based on the outcome of conversations with characters in the scene. Covert Protocol unlocks social simulation game mechanics with AI Digital Humans acting as custodians of crucial information, presenting challenges, and catalyzing key narrative developments. Digital humans play a critical role in the experience.
Powered by the Inworld Engine and leveraging NVIDIA ACE, each player's journey through Covert Protocol is unique. Players’ real-time decisions and strategic planning lead to different game outcomes, ensuring that no two playthroughs are alike. This level of AI-driven interactivity and player agency opens up new possibilities for emergent gameplay, where players must think on their feet and adapt their strategies in real-time to navigate the intricacies of the game world.
Built on Unreal Engine 5, Covert Protocol leverages Inworld’s SDK to make integrating with MetaHuman fast and simple, and incorporates NVIDIA ACE technologies, including NVIDIA Riva Automatic Speech Recognition (ASR) and NVIDIA Audio2Face (A2F) to augment Inworld’s speech and animation pipelines.
“The combination of NVIDIA ACE microservices and the Inworld Engine enables developers to create digital characters that can drive dynamic narratives, opening new possibilities for how gamers can decipher, deduce and play.” said Kylan Gibbs, CEO of Inworld AI.
To accelerate developer adoption, Inworld will be releasing Covert Protocol’s source code in the near future, enabling developers to learn from the tech demo, and use it to create their own innovations.
World of Jade Dynasty & Unawake Also Incorporating Audio2Face Into Development Pipeline
Perfect World Games’ World of Jade Dynasty is an upcoming Unreal Engine 5 massively multiplayer online role-playing game, and sequel to the firm’s incredibly popular Jade Dynasty.
A new video captured in-game demonstrates offline-rendered NVIDIA Audio2Face technology that generates accurate lip syncing in both Chinese and English, allowing script writers to easily add new voiced content to World of Jade Dynasty. Audio2Face can be used to easily support multi-language versions of the game.
In RealityArts Studio and Toplitz Productions’ Unreal Engine 5-powered Unawake, you decide the fate of humanity amidst the turmoil of merciless wars between nightmarish creatures from Heaven and Hell. Deliver punishment or absolution to your opponents in this action-adventure as you fight them in intense first-person melee skirmishes. You are the last bastion.
Unawake leveraged Audio2Face to create the characters’ facial animations during gameplay and in high-fidelity cinematics. Watch the new Unawake video below to see Audio2Face in action, and stay tuned for the game’s release later this year also featuring DLSS 3 with Frame Generation.
NVIDIA ACE Technologies: Building Blocks For Digital Humans and Virtual Assistants With Hippocratic AI & UneeQ
To showcase the new capabilities of NVIDIA digital human technologies, we’ve worked with leading companies, including Hippocratic AI and UneeQ, on a series of new technology demonstrations.
HippocraticAI has created a safety-focused, LLM-powered, task-specific Healthcare Agent. The agent calls patients on the phone, follows up on care coordination tasks, delivers preoperative instructions, performs post-discharge management and much more. For GTC, NVIDIA collaborated with HippocraticAI to extend its solution to use NVIDIA ACE microservices, NVIDIA Audio2Face along with NVIDIA Animation graph and NVIDIA Omniverse Streamer Client to show the potential of a generative AI healthcare agent avatar.
“Our digital assistants provide helpful, timely and accurate information to patients worldwide,” said Munjal Shah, cofounder and CEO of Hippocratic AI. “NVIDIA ACE technologies bring them to life with cutting-edge visuals and realistic animations that help better connect to patients.”
UneeQ is an autonomous digital human platform specialized in creating AI-powered avatars for customer service and interactive applications. UneeQ’s digital humans represent brands online, communicating with customers in real time to give them confidence in their purchases. UneeQ integrated NVIDIA Audio2Face microservice into its platform and combined it with Synanim ML to create highly realistic avatars for a better customer experience and engagement.
“UneeQ combines NVIDIA animation AI with our own Synanim ML synthetic animation technology to deliver real-time digital human interactions that are emotionally responsive and deliver dynamic experiences powered by conversational AI,” said Danny Tomsett, CEO and Founder at UneeQ.
NVIDIA is at the forefront of Generative AI, and developers and platforms embracing NVIDIA ACE include: Convai, Cyber Agent, Data Monsters, Deloitte, HippocraticAI, IGOODI, Inworld AI, Media.Monks, miHoYo, NetEase Games, Perfect World, Openstream, OurPalm, Quantiphi, Rakuten Securities, Slalom, SoftServe, Tencent, Top Health Tech, Ubisoft, UneeQ and Unions Avatars. Head to their websites to discover how they’re leveraging digital human technology to create new possibilities and enhance customer experiences.
NVIDIA ACE: Bring Digital Characters To Life
Top game and digital human developers are pioneering ways ACE and generative AI technologies can be used to transform interactions between players and NPCs in games and applications. Learn more about NVIDIA ACE, future developments, and early access programs. And discover more uses of NVIDIA RTX technology and GeForce RTX GPUs in the latest games by heading to GeForce.com.