Cyberpunk 2077 is just a few weeks away and the action-packed RPG will launch with support for dialogue in 10 languages and subtitle options for several others. CD Projekt Red is aiming to add a deeper layer of immersion for players by using artificial intelligence to lip sync the dialogue in multiple languages.
The game’s lead character technical director, Mateusz Popławski, said in a presentation (via @shinobi602 on Twitter) that CDPR targeted better lip sync quality than in The Witcher 3. It wanted to do so in 10 languages: English, German, Spanish, French, Italian, Polish, Brazilian Portuguese, Russian, Mandarin and Japanese.
The goal was to do all of that for every character in the open world. Because of the game’s enormous scope, CDPR needed to do all of that with zero facial motion capture. To do so, the studio tapped into Jali Research’s lip syncing and facial animation tech to procedurally generate how the characters’ faces move.
Cyberpunk 2077 has full lip-sync for every character in all 10 dubbed languages
Presentation: https://t.co/8r7rq570VM
Nuts – and awesome to see for languages that often times get much less support. Great work by everyone involved. pic.twitter.com/hwvaTWb9ei
— Shinobi602 (@shinobi602) October 20, 2020
Some of the characters in Cyberpunk 2077 speak multiple languages, sometimes switching between them in the same sentence. To account for that, transcript tagging was used. The tags also helped to adjust a character’s facial expressions when their emotional state changed within a line of dialogue. The system also used audio analysis to replicate the emotions of a vocal performance in the animations.
CDPR used algorithms to help it animate the cutscenes in The Witcher 3, and this is an evolution of that approach. Many games use motion capture to lip sync dialogue in just one language, which can break the immersion a bit for players who switch to a different one.
While the procedurally generated animations in Cyberpunk 2077 might not be quite as detailed or as expressive as those in some other AAA games, they could add to the experience for many players who’d prefer to play in different languages. You’ll be able to take a closer look at how that works in practice when the game arrives on November 19th.
Author: Kris Holt, @krisholt
2h ago
Source: Engadget
