voice-clone-and-lip-sync

A pipeline for voice-cloning and lip-syncing videos using Open-source tools.

Tech stack:

1. Ffmpeg: Audio extraction and minor edits
2. Whisper: For audio transcription
3. 🐸xTTS: A multi-lingual voice-cloning end-to-end model from Coqui-ai
4. Wav2lip: For lip-syncing

In Action

Trump's original speech

Trump.says.he.can.end.Ukraine.war.in.a.day.mp4

Trump's speech in Hindi

trump-hindi.mp4

Trump's speech in French

trump-french.mp4