Capacitron

[a-froghyar]

This PR implements a new model into 🐸 TTS based on the Capacitron model from Google. It's a partial implementation of the models detailed in the paper, hierarchical latent embeddings are still to be done - this is a TODO for later. If you'd like to get an idea what the model does and how it works, here's a post I did a few months ago.

I have implemented this model as part of my Master's Thesis at TU Berlin. The thesis itself is a detailed report on the implementation and subjective evalutation of this model. You can read my thesis and listen to some samples here. I'm in the process of creating a website with audio samples from my pretrained models and the uploaded thesis as well - this is WIP.

I have implemented this model into an earlier version (March 2020) of 🐸 TTS, so this new "re-implementation" still needs to be tested. I'm in that process right now, however I've wanted to open this PR already to discuss some of the ways the Trainer API needs to be adjusted to accomodate the model.

TODOs:

• Fix tests
• Create T1 Capacitron Test
• Create T2 Capacitron Test
• Tokenizer update, delete explicit espeak flag
• Pull out gradient clipping
• Open PR in the trainer
  |-----> Implement 2 model specific methods Trainer#26
• Test Tacotron 1 Model
• Test Tacotron 2 Model (thanks to help from @Edresson, the Capacitron VAE module is modular, so we could test it in T2 as well, however the original authors did not explore this for some reason)
• Test different attention algos [in my experiments, only graves worked]
  - original doesn't work with capacitron and dynamic_convolution only works with Tacotron 2
  - Use Graves with T1
  - User DCA with T2
• Fix beta initialisation on continue trainings

@erogol @Edresson @WeberJulian I'd appreciate it if you could review the changes and discuss some of the specifics in the code.

[a-froghyar]

Update: I've just ran the first training and it is slightly off. I suspect there's an error in the loss calculation because of the reorganisations from @Edresson a few months back. I'm investigating that today and will push the new commits.

[a-froghyar]

Update: managed to do the first training that gave some promising results. I'm including now a step wise gradual lr scheduler (unlike the Noam Scheduler, this takes in hardcoded step # thresholds and learning rates), which proved necessary in my previous implementation. More updates to follow. :)

[a-froghyar]

This file will be deleted before the merge, it's only a script to show how to infer the model for others who are experimenting with this work

[a-froghyar]

@erogol from my side this is ready to go. 😊 Big thanks to @WeberJulian for all the help!

[WeberJulian]

Just waiting for my ljpeech T2 capacitron run to converge. If it does, I'll merge both this PR and coqui-ai/Trainer#26

[a-froghyar]

Trainings are not converging since the reorganisation of the previous 2 weeks. Reporting back here soon

Edit: commits below fixed all issues

[erogol]

FINALLY !!!! 🚀