ElevenLabs announce AI Dubbing – the latest in a line of ground-breakingly high-quality AI audio tools. The AI start-up is already a familiar face in the consumer AI space, thanks to its viral voice cloning tool which dominated TikTok earlier this year. With a specialist focus on audio AI research and development, what can we expect from the new feature?
What is Eleven Labs AI Dubbing feature?
The new Eleven Labs AI Dubbing feature can translate any digitally recorded speech into any other language (from an ever-expanding list). In the LinkedIn announcement shared earlier today, the firm announced the following features:
- Over 20 languages at launch
- Automatic detection of multiple speakers
- Background noise separated from voices
In its own words, “ElevenLabs started with the mission to break down language barriers, and today, we are a step closer! Convert spoken content to other languages in minutes, while preserving all of the original voices.”
With our AI Dubbing feature – our biggest milestone yet – creators and their audiences can now enjoy the best of both worlds: global reach & authenticity.
ElevenLabs, LinkedIn announcement
Essential AI Tools
Content Guardian – AI Content Checker – One-click, Eight Checks
Jasper AI
WordAI
Copy.ai
Writesonic
The idea here is that your accent, intonation, inflections and so on are retained. The USP here is that each dub (you can think of it as a translation) still sounds like your own voice, but in a different language. Traditionally, shows are dubbed into other languages so that a wider audience can enjoy them.
This is profitable because, compared to the six or seven-figure sum that was spent on the entire piece of video content (e.g a Netflix show) dubbing is very cost effective per additional viewer. Even so, this process typically involves a translator and a voice actor – a slow and multi-stage process that also includes hiring two additional professionals and renting a VO studio. The costs only snowball when hiring several voice actors in the case of dubbed reality TV shows. The cost factor is very pertinent, as you’ll see in a minute.
What languages does ElevenLabs AI Dubbing support?
The listed of currently available languages includes German, Spanish, French, Italian, Hindi, Korean, Dutch, Turkish, Swedish, Indonesian, Vietnamese, Filipino, Ukrainian, Greek, Czech, Finnish, Romanian, Danish, Bulgarian, Malay, Hungarian, Norwegian, Slovak, Croatian, Classic Arabic, Tamil, English, Polish, and Portuguese.
Voice clone, voice design, voice dubbing – What’s next for AI audio?
This process is set to change dramatically. In the near future, dubbing will take place in the cutting room. Editors will be able to translate actors, hosts, and contestants etc. without themselves knowing either language. If you’ve ever seen subtitles that say something along the lines of “Speaking Swedish” now you know why; The job of a translator was expected to be miraculously completed by an editor. The best editor for the job does not necessarily speak any given language – and now that’s no longer a problem.
How to use AI Dubbing
Content creators will benefit greatly from this, for multiple reasons.
- Dubbing content allows you to reach a wider global audience, and increase viewership on the same content. This means no additional time investment, but higher returns (viewership and revenue). The cost factor has always been prohibitively expensive for most creators, but now anyone can afford to create content in multiple languages.
- Content creators with an archive of older content can be translated and republished at a fraction of the time it took to create the original content.
- Your own voice is preserved. Unlike other solutions, such as outsourcing VO, the end result sounds like you – the voice your audience recognises and is already familiar with. From a purely analytic perspective, this also removes one variable from performance analysis; It could be that the voice actor you hire has a disagreeable voice to some people, causing them to turn off the video and impact your watch time. You’ll never know if that’s the case, because the exiting viewer is not leaving feedback. It doesn’t even matter how unlikely this scenario is – the fact is that one less variable is a good thing.
With a higher-quality replication of human speech than any competitor, ElevenLabs is dominating the audio AI space for all the right reasons. High-quality output combined with automatic dubbing is sure to be a staple tool of content creation in the coming years.
When and where can you get it?
The new feature is accessible right now, right here.
ElevenLabs focus is “to make content universally accessible in any language & voice”. Able to accurately present a wide range of emotions, regardless of the users voice type or gender, this advanced AI technology is set to make AI narration as well as all manner of audio content production a breeze.
