Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TTS FastPitch Finetuning #2684

Merged
merged 22 commits into from
Aug 26, 2021
Merged

TTS FastPitch Finetuning #2684

merged 22 commits into from
Aug 26, 2021

Conversation

paarthneekhara
Copy link
Collaborator

No description provided.

blisc
blisc requested changes Aug 18, 2021
View reviewed changes
examples/tts/conf/fastpitch_align_44100.yaml Outdated Show resolved Hide resolved
examples/tts/conf/fastpitch_align_finetuning.yaml Outdated Show resolved Hide resolved
nemo/collections/asr/data/audio_to_text.py Outdated Show resolved Hide resolved
nemo/collections/tts/models/fastpitch.py Show resolved Hide resolved
nemo/collections/tts/models/fastpitch.py Outdated Show resolved Hide resolved
tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb Outdated Show resolved Hide resolved
tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb Outdated
"from nemo.collections.tts.models import HifiGanModel\n",
"from nemo.collections.tts.models import FastPitchModel\n",
"\n",
"hifigan_ckpt_path = \"/home/pneekhara/PreTrainedModels/HifiGan--val_loss=0.08-epoch=899.ckpt\"\n",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably also need to publish this model to NGC

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, will need to do that.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a placeholder for the FastPitch checkpoint as well, and add a comment that we plan on releasing .nemo files for this notebook soon.

tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb Show resolved Hide resolved
tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb Show resolved Hide resolved
tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb Show resolved Hide resolved
blisc
blisc reviewed Aug 19, 2021
View reviewed changes
tutorials/tts/4_TTS_FastPitch_Finetuning.ipynb
" plt.show()"
]
},
{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would also be cool to show how people can fine tune their model on other data. Maybe even use their own voice after recording a few samples.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can move this to another PR

@blisc blisc mentioned this pull request Aug 19, 2021
Merged
@blisc blisc self-assigned this Aug 19, 2021
@paarthneekhara
Added speaker in FastPitch2 dataloader
1383553 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
changed fastpitch_align.yaml to have configuration for 441000 Hz
2d68e76 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
trying to fix g2p
fad6d3a 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
pytorch lightning version requirement
abff093 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
Cleanupfinetuning (#2)
0017e06 
* Added filelists and finetuning code

* image logging transoposed bug fix in FastPitch

* pytorch lightning version requirement

* added filelists for multispeaker hifigan

* hifigan 44100 Hz

* updated filelists, added another configuration of hifigan generator as per jason

* added notebooks and some scripts

* synthesize samples script update

* removed sample filelists

* reverted hifigan changes

* reverting hifigan.yaml update

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
added finetuning notebook
f91a9e0 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
removed fastpitch2.py (redundant with fastpitch.py)
93c4379 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
restored old fastpitch_align.yaml, made just one finetuning yaml
715ad0c 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
reverting to old fastpitch align yaml
bd06780 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
reverted to old vocabs.py
b97b095 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
reverted to old vocabs.py
a751785 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
reverting requirements change
1b5291f 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
addressed pull request reviews -- updated notebook, speaker loading f…
68cb863 
…or fastpitch dataset, comments in configuration files

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
notebook update
6d53329 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
removed redundant configuration file, updated notebook
d917be3 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
some more corrections for switching to single configuration file
322db0c 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
added url for hifigan dataset in notebook
bf78860 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
warning messages if optimizer configuration does not look correct whi…
8a6f8b0 
…le finetuning

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@paarthneekhara
style error fix
e9dfa32 
Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>
@blisc blisc merged commit 0d7de7c into NVIDIA:main Aug 26, 2021
jfsantos pushed a commit to jfsantos/NeMo that referenced this pull request Nov 19, 2021
@paarthneekhara @blisc @jfsantos
TTS FastPitch Finetuning (NVIDIA#2684)
3e8aedd 
* Added speaker in FastPitch2 dataloader

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* changed fastpitch_align.yaml to have configuration for 441000 Hz

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* trying to fix g2p

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* pytorch lightning version requirement

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* Cleanupfinetuning (NVIDIA#2)

* Added filelists and finetuning code

* image logging transoposed bug fix in FastPitch

* pytorch lightning version requirement

* added filelists for multispeaker hifigan

* hifigan 44100 Hz

* updated filelists, added another configuration of hifigan generator as per jason

* added notebooks and some scripts

* synthesize samples script update

* removed sample filelists

* reverted hifigan changes

* reverting hifigan.yaml update

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* added finetuning notebook

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* removed fastpitch2.py (redundant with fastpitch.py)

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* restored old fastpitch_align.yaml, made just one finetuning yaml

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* reverting to old fastpitch align yaml

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* reverted to old vocabs.py

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* reverted to old vocabs.py

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* reverting requirements change

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* addressed pull request reviews -- updated notebook, speaker loading for fastpitch dataset, comments in configuration files

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* notebook update

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* removed redundant configuration file, updated notebook

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* some more corrections for switching to single configuration file

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* added url for hifigan dataset in notebook

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* warning messages if optimizer configuration does not look correct while finetuning

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* style error fix

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* dataloader fix after master merge

Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

Co-authored-by: Jason <jasoli@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@blisc blisc blisc approved these changes

Assignees

@blisc blisc

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@paarthneekhara @blisc