Captions, an AI-powered video modifying startup, has launched a brand new app known as Lipdub for translating clips into 28 totally different languages.
Lipdub is accessible within the App Retailer free of charge and helps a number of languages together with French, Hindi, Spanish, Italian, Portuguese, Japanese, and extra. The app even lets customers translate movies into Texas slang, Gen Z, pirate, and child speak. The demo video exhibits that the app may also change lip motion based on the chosen goal language. Nonetheless, at instances there’s a sure lag between the audio and the lip motion.
Customers can translate a video of a single individual speaking throughout as much as one minute after which share it on different social media platforms.
On its web site, Captions says that greater than 3 million creators have used its eponymous video modifying app. The startup claims that it has greater than 100,000 every day customers. The Captions app affords a number of AI-powered options round video modifying, comparable to eradicating “hums” and “ahs”, decreasing background noise and enhancing speech. The app additionally has an “AI Lipdub” characteristic that may change lip motion in post-production modifying when you change the transcript.
Captions was based in 2021 by Gaurav Misra, who was head of design engineering at Snap in his earlier function. In June, the corporate secured $25 million in a Sequence B spherical led by Kleiner Perkins with participation from Sequoia Capital, Andreessen Horowitz (a16z), and SV Angel. To this point, Captions has raised $40 million in funding.
Utilizing translation and AI-dubbing to succeed in a wider viewers is a rising pattern. In June, YouTube introduced that it’s testing an AI-powered device to let customers robotically dub their movies in different languages. The corporate stated it’s even engaged on higher lip-syncing. Final month, the corporate stated that it’s integrating AI-powered dubbing instantly into YouTube Studio for simpler entry for individuals trying to convert movies into different languages.
Earlier this month, AI-powered voice-generating platform ElevenLabs additionally launched its dubbing device with help for 29 languages. Remainder of World beforehand reported that dubbing service supplier firms are producing tens of millions of {dollars} by translating content material for widespread YouTubers like MrBeast.
AI-powered dubbing startups have generated a number of investor curiosity with startups like UK-based Papercup and Isreal-based Deepdub elevating tens of millions of {dollars}.