Deepdub, an AI-based audiovisual dubbing and language localization company, has announced the launch of Deepdub Go, a way for creators to dub in 65 languages.
The new AI-powered audio-video localization and creation platform is aimed at indie game studios, advertising agencies, online learning platforms, and content creators. Deepdub Go enables creators to use their own voice for dubbing and to control vocal emotional expression using emotion-prompting AI.
The platform delivers good voice quality and emotional resonance, equipped with human-like sound, emotion-prompting and voice-guiding technology, and a robust composer for high-quality voice cloning, the company said. Tel Aviv-based Deepdub said the professional platform’s capabilities extend to editing voice characteristics, languages, and translations, in addition to other powerful editing features such as controlling duration and word count for lip-sync adjustment.
Also, the platform enables control through “voice guiding” — a technique developed by Deepdub to achieve theatrical-grade results for Hollywood movies. Users can provide a voice recording to act as a guide for the desired emotional expressivity of the AI-generated audio.
“We want to democratize the way people localize content and democratize the global outreach of content to make it accessible around the globe,” said Ofir Krakowski, CEO and cofounder of Deepdub, in an interview with GamesBeat.
Deepdub initially developed its proprietary AI technology for the entertainment industry, collaborating with Hollywood studios on dozens of film and television projects. With the launch of Deepdub Go, the company is now bringing high-quality and scalable video localization to a wider range of industries for the first time, enabling creators to make their content accessible to a wider audience.
Users can upload their videos to the Deepdub Go platform, choose the video style and desired languages, and the video is available for download shortly. The end-to-end process includes automatic transcription, translation, voice generation, and audio mixing. Deepdub Go is also available as an application programming interface (API), enabling integration into existing tools and automated workflows.
“For the first time ever, professional-level dubbing is available for any type of content, whether it’s a corporate training video, a university course, or an influencer’s video,” said Krakowski. “Deepdub Go is an enabler of global storytelling, allowing content creators to resonate with their audiences in a deeper, more personal way. With our studio platform, creators can amplify the potential reach of every piece of content beyond traditional geographical and linguistic boundaries.”
Deepdub wants to bridge the language barrier and cultural gap in entertainment experiences for international audiences across TV, film, ads, gaming, and e-learning. Films using it are currently streaming on Hulu and Amazon Prime.
Krakowski has 30 years of experience in computer science, and he was head of machine learning and innovation for the Israeli Air Force prior to working in the industry. He started the company with his younger brother Nir in 2019. The company focused on Hollywood studios and production houses at first.
“Deepdub aims to breach language barriers, the cultural gaps,” he said. “We’re democratizing this by launching the Deepdub Go platform to the public. We’re aiming to impact the global audience and enable people access to content across different regions.”
Before technologies like ChatGPT came along, the company had to develop its own AI training platform and its own voice models to get high-quality translation. they found that pure machine translation could get them to 70% to 90% accuracy. To get higher than that, they needed a human to fix it, Krakowski said. Translating jokes using AI was particularly hard.
“In creating emotions, actually, our technology can understand that,” he said. “As a person, you can go and say I want it to be more funny or less funny or sarcastic, so you can actually control the level of emotion.”
You can still use human intervention, but now the tech is better, Krakowski said. In games, this is particularly useful as there can be hundreds of characters with different voices in a single game. With phrases using idioms, sometimes there isn’t a good translation. But the company has made its own attempts to translate idiomatic phrases so they can make sense.
There are still a lot of languages to add. In India, for instance, there are more than 100 languages. The company has 33 full-time employees and hundreds more freelancers around the world.
Missed the GamesBeat Summit excitement? Don’t worry! Tune in now to catch all of the live and virtual sessions here.
Deepdub, an AI-based audiovisual dubbing and language localization company, has announced the launch of Deepdub Go, a way for creators to dub in 65 languages.
The new AI-powered audio-video localization and creation platform is aimed at indie game studios, advertising agencies, online learning platforms, and content creators. Deepdub Go enables creators to use their own voice for dubbing and to control vocal emotional expression using emotion-prompting AI.
The platform delivers good voice quality and emotional resonance, equipped with human-like sound, emotion-prompting and voice-guiding technology, and a robust composer for high-quality voice cloning, the company said. Tel Aviv-based Deepdub said the professional platform’s capabilities extend to editing voice characteristics, languages, and translations, in addition to other powerful editing features such as controlling duration and word count for lip-sync adjustment.
| Read GamesBeat’s special issue, Gaming communities: Making connections and fighting toxicity. |
Also, the platform enables control through “voice guiding” — a technique developed by Deepdub to achieve theatrical-grade results for Hollywood movies. Users can provide a voice recording to act as a guide for the desired emotional expressivity of the AI-generated audio.
Event
Transform 2023
Join us in San Francisco on July 11-12, where top executives will share how they have integrated and optimized AI investments for success and avoided common pitfalls.
“We want to democratize the way people localize content and democratize the global outreach of content to make it accessible around the globe,” said Ofir Krakowski, CEO and cofounder of Deepdub, in an interview with GamesBeat.
Deepdub initially developed its proprietary AI technology for the entertainment industry, collaborating with Hollywood studios on dozens of film and television projects. With the launch of Deepdub Go, the company is now bringing high-quality and scalable video localization to a wider range of industries for the first time, enabling creators to make their content accessible to a wider audience.
Users can upload their videos to the Deepdub Go platform, choose the video style and desired languages, and the video is available for download shortly. The end-to-end process includes automatic transcription, translation, voice generation, and audio mixing. Deepdub Go is also available as an application programming interface (API), enabling integration into existing tools and automated workflows.
“For the first time ever, professional-level dubbing is available for any type of content, whether it’s a corporate training video, a university course, or an influencer’s video,” said Krakowski. “Deepdub Go is an enabler of global storytelling, allowing content creators to resonate with their audiences in a deeper, more personal way. With our studio platform, creators can amplify the potential reach of every piece of content beyond traditional geographical and linguistic boundaries.”
Deepdub wants to bridge the language barrier and cultural gap in entertainment experiences for international audiences across TV, film, ads, gaming, and e-learning. Films using it are currently streaming on Hulu and Amazon Prime.
Origins
Krakowski has 30 years of experience in computer science, and he was head of machine learning and innovation for the Israeli Air Force prior to working in the industry. He started the company with his younger brother Nir in 2019. The company focused on Hollywood studios and production houses at first.
“Deepdub aims to breach language barriers, the cultural gaps,” he said. “We’re democratizing this by launching the Deepdub Go platform to the public. We’re aiming to impact the global audience and enable people access to content across different regions.”
Before technologies like ChatGPT came along, the company had to develop its own AI training platform and its own voice models to get high-quality translation. they found that pure machine translation could get them to 70% to 90% accuracy. To get higher than that, they needed a human to fix it, Krakowski said. Translating jokes using AI was particularly hard.
“In creating emotions, actually, our technology can understand that,” he said. “As a person, you can go and say I want it to be more funny or less funny or sarcastic, so you can actually control the level of emotion.”
You can still use human intervention, but now the tech is better, Krakowski said. In games, this is particularly useful as there can be hundreds of characters with different voices in a single game. With phrases using idioms, sometimes there isn’t a good translation. But the company has made its own attempts to translate idiomatic phrases so they can make sense.
There are still a lot of languages to add. In India, for instance, there are more than 100 languages. The company has 33 full-time employees and hundreds more freelancers around the world.
GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it. Discover our Briefings.
Author: Dean Takahashi
Source: Venturebeat
