[Feature] Generating a vocal time code file for karaoke

[bigboss97]

Description

I don't know how this software work. But it's simply amazing! :-)
I would love to use it for creating karaoke music file, i.e. I have to synchronize the lyrics. I'm wondering whether it's possible to generate the time code information while splitting the vocal, of course not each single words, but between silence and vocals.
For instance, the software generates a text file with the timestamps of vocal starts and ends.

[mmoussallam]

Hi,

What you're suggesting is actually quite hard, it's called lyrics synchronisation and is still nowadays an active research topic in music analysis.

Isolating vocals with spleeter is definitely something that should make this task easier, but you need other tool to to the synchronisation, ones that we don't have right now.

You can take a look at the systems proposed in the MIREX competition: https://www.music-ir.org/mirex/wiki/2019:Automatic_Lyrics-to-Audio_Alignment to get a glance of the latest work available.

[bigboss97]

I totally understand that. I don't expect automatic lyric synchronization. spleeter doesn't have to care the lyrics. It only has to mark between vocal and silence of the vocal track with timestamps. Then this information can be imported to a K-editor and the human can paste the text between the timestamps. It's not fully automated. But it would help a lot in the manual work.

Thanks for the link, very interesting.