Q: You've mentioned using Eleven Labs voices.
I use Wayne in Multilingual V1 setting on Eleven Labs to read text in Hindi script mixed with Roman Hindi. Eleven Labs does a flawless job of it. Would I be able to do the same in Dubverse?
Tanay_Dubverse
Jun 27, 2024A: Hi sanaisme,
As already pointed out by Varshul and Anuja, we are adding more speakers from Eleven Labs to Dubverse. In the next iteration, we would also incorporate the multilingual capabilities of those speakers in the product.
Hope this helps.
We currently have two English speakers on Elevenlabs present on Dubverse, Josh and Bella, you can use them only when you buy the plan.
We will keep adding new voices from Elevenlabs to the Dubverse platform but we do not support all voices for now.
Hi, we currently we support Josh and Bella from elevenlabs. They work in English.
We are coming with more elevenlabs voices and would support multi-lingual versions both V1 and their recent V2 models.
For now you can try our Hindi voices which support mix languages. Do try out Shan and Sunidhi :)
Verified purchaser
Thanks for the heads up. I want to buy the highest tier but will wait as my use case requires Eleven Labs Multilingual V1 and V2. The tts output is for an Urdu audience. However, since I am more fluent in Hindi than Urdu script, I can spot and correct mistakes better when the text is in Hindi. So Eleven Labs ignores the script being Hindi, instead it reads the text with Urdu mannerism because the vocabulary in the text is Urdu. How awesome is that! The intention is to create an Urdu version of the Qur'an with a male voice. Hence the choice of highest tier. If you get the voice of Wayne from Eleven Labs in Dubverse, my world will be a tad more sparkly.
Verified purchaser
I am testing Dubverse via the trial on the website. Here's what's happened. It's done a good job in terms of render speed, and that's a whopping achievement. Dubbed video of a girl speaking in English. Chose female voice for dubbing in Urdu. The dubbed voiceover adopts masculine gender. For example, the dubbed version says - 'main gaya' for 'I went', when it should be 'main gayi'. Wouldn't selecting a female voice indicate to the algorithm about the gender? As a South Asian, you know that in English, active verbs are devoid of gender. But in Urdu/Hindi, active verbs have masculine or feminine gender.
2. The lady in the original video is speaks very fast and Dubverse has done a good job of syncing the dub. There are suggestions in the subtitle section to fix the dub overlap issue when CPS is higher than optimal. Is there a video to explain it in detail, as I could not figure out how to extend the length of the segment in the timeline.
3. What's the best practice for videos where the CPS is high? For example, should the original video be downloaded, optimized for playback speed and uploaded to Dubverse?
4. Why do the Urdu voices say 'heylo' instead of 'hello'? I believe this has nothing to do with Dubverse. Would changing the subtitle text as per acoustics in Dubverse fix it?
thank you for all the kind words :)
1. about male active verbs, yes the AI is still biased towards the male gender, we have added fine-tuning to make it more gender-neutral but some work needs to be done here. For now, you need to correct these manually under studio > redraft. You can use Find and Replace to make changes at multiple places with a single click.
2. yes please watch the video here on how to use resync mode: https://www.youtube.com/watch?v=pBJ_ZQV0aM4
the other alternative is to edit the translation of the specific segment from redraft mode and reduce the count of words, this will in turn reduce CPS and speed of the segment in the video
3. this is the most complex issue when it comes to language translation, for now with you can use resync to make edit to the time stamps or use redraft to shorten the translation generated to get the ideal CPS count.
working on more solutions here currently with the AI team, to auto-correct the CPS by tweaking the translation
4. yes correct. this seems to be misspelt in the translation, Please edit from the redraft more, and you can listen to the specific segment from the retune mode to make sure you got the right pronunciation before you reprocess video to make changes
hope this helps, and we will see you creating more urdu videos from dubverse :)