TTS recipe

[jimbozhang]

I'm not sure if it useful to let lhotse support TTS training. If needed, I can make a TTS recipe using the LJSpeech corpus.
@danpovey @pzelasko

[danpovey]

That sounds like a great idea! Yes, I intend for it to support all kinds of speech-related tasks.

…

On Tue, Sep 1, 2020 at 2:52 PM Junbo Zhang ***@***.***> wrote: I'm not sure if it useful to let lhotse support TTS training. If needed, I can make a TTS recipe using the LJSpeech <https://keithito.com/LJ-Speech-Dataset/> corpus. @danpovey <https://github.com/danpovey> @pzelasko <https://github.com/pzelasko> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#67>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZFLO6M6P5F2PDHCW2PS2TSDSK4JANCNFSM4QRLERMQ> .

[pzelasko]

Yes, it will be most useful. Thanks!

[jimbozhang]

Just started working on this. A new recipe ljspeech will be added for end-to-end (e.g. Tacotron2) training. I'll make a PR next week.

[pzelasko]

Sounds great! A heads up - I made a couple of general changes in the meanwhile, e.g. the preferred manifest format is now JSON because it’s much faster to read/write.

…

On Sep 9, 2020, at 09:59, Junbo Zhang ***@***.***> wrote: Just started working on this. A new recipe ljspeech will be added for end-to-end (e.g. Tacotron2) training. I'll make a PR next week. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#67 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZRKQBOF4CUW4D3KADCTNDSE6C2VANCNFSM4QRLERMQ>.

[jimbozhang]

Sounds great! A heads up - I made a couple of general changes in the meanwhile, e.g. the preferred manifest format is now JSON because it’s much faster to read/write.

I had a little worry: if the dataset is extremely huge, the JSON would be very large. To avoid out-of-memory, the manifest has to be splitted. I think splitting YAML is easier than JSON, because if you want to split a JSON, you have to load it into memory first.

[danpovey]

We had also discussed JSONL format (JSON, one element per line). It will take quite a big dataset to hit that memory limit though.

…

On Wed, Sep 9, 2020 at 10:28 PM Junbo Zhang ***@***.***> wrote: Sounds great! A heads up - I made a couple of general changes in the meanwhile, e.g. the preferred manifest format is now JSON because it’s much faster to read/write. I had a little worry: if the dataset is extremely huge, the JSON would be very large. To avoid out-of-memory, the manifest has to be splitted. I think splitting YAML is easier than JSON, because if you want to split a JSON, you have to load it into memory first. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#67 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZFLO4BVH64RAQJQWYARODSE6GKVANCNFSM4QRLERMQ> .

[pzelasko]

Hmm, you're raising a good point. We can add JSONL quite easily - we're already supporting two formats anyway...

I wonder if it makes sense to use a binary format for such super-large datasets. E.g. in https://github.com/huggingface/nlp/ they use Apache Arrow to traverse mmap-ed files, allowing them to iterate very large datasets with almost zero memory footprint (likely at the cost of storage size).

[danpovey]

Lets solve that only once it becomes a problem.

…

On Wed, Sep 9, 2020 at 11:00 PM Piotr Żelasko ***@***.***> wrote: Hmm, you're raising a good point. We can add JSONL quite easily - we're already supporting two formats anyway... I wonder if it makes sense to use a binary format for such super-large datasets. E.g. in https://github.com/huggingface/nlp/ they use Apache Arrow to traverse mmap-ed files, allowing them to iterate very large datasets with almost zero memory footprint (likely at the cost of storage size). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#67 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZFLOZACYMLDBKYPYFV7ETSE6KCVANCNFSM4QRLERMQ> .