Localization is a tricky issue for all content creators. It can take significant time and resources to make their work fully accessible to folks who speak different languages. One company thinks it has cracked part of that code with an artificial intelligence system that automatically translates speech into other languages in the same speaker’s voice.
Resemble AI says its Localize tool can keep voices consistent in various languages in movies, games, audiobooks, corporate videos and other formats. Google is working on similar tech, but we haven’t heard much about that since it published a paper on the Translatotron system last year.
The implications of Localize are intriguing. For instance, if it translated audio from Dune, Timothée Chalamet could sound like himself even in the German or Italian versions — whenever that movie actually makes it to theaters, anyway.
YouTubers might create versions of their videos for fans who otherwise rely on captions to understand them. Call centers could create chatbots that support multiple languages. All of that’s assuming Localize works as well as intended, though.
For now, Localize can translate speech between English, French, German, Dutch, Italian and Spanish. There are plans to add Korean, Japanese and Mandarin to the mix in the near future.
Resemble AI says Localize can translate recordings in a way that accurately reflects the speaker&aposs words and meanings. The system, it claims, can turn the original audio into speech that uses colloquialisms and grammar structures of a certain region and language.
One major downside of this tech is that, if it works as well as Resemble AI suggests, actors could miss out on a lot of dubbing work. Given delicate nuances in certain languages, it would probably be best for a native speaker to review the translations as well. It wouldn’t be a great look if Keanu Reeves accidentally insulted all of the Netherlands in Cyberpunk 2077, for instance. Still, if Localize works as promised, it could allow creators of audio-based media to share their work with more fans with greater ease.
Author: Kris Holt, @krisholt
1h ago
Source: Engadget
