Recommended audio parameters for training HifiGan for tts_models/ja/kokoro/tacotron2-DDC ?

[kaiidams]

I'm trying to train HifiGan for tts_models/ja/kokoro/tacotron2-DDC as the default vocoder WaveGrad is a bit slow on CPU.
I started finetuning from vocoder_models/en/vctk/hifigan_v2. Though it has different audio parameters than the TTS model,
I thought some of weights are useful.

The model that was trained for 355000 steps, can generate realistic audio, but with some artifacts. It seems the model is not improving much after around 200000.

This colab converts a true audio with ap.melspectrogram() and then converts back with vocoder_model.inference().
https://colab.research.google.com/drive/1xB-rDm6NUrWNUGPEWaAgrb6PFcMfSOpM?usp=sharing
Audios are also attached in this discussion.
samples.zip

The audio parameters I use is,

 > Setting up Audio Processor...
 | > sample_rate:22050
 | > resample:False
 | > num_mels:80
 | > min_level_db:-100
 | > frame_shift_ms:None
 | > frame_length_ms:None
 | > ref_level_db:20
 | > fft_size:1024
 | > power:1.5
 | > preemphasis:0.0
 | > griffin_lim_iters:60
 | > signal_norm:True
 | > symmetric_norm:True
 | > mel_fmin:50.0
 | > mel_fmax:7600.0
 | > spec_gain:1.0
 | > stft_pad_mode:reflect
 | > max_norm:4.0
 | > clip_norm:True
 | > do_trim_silence:True
 | > trim_db:60
 | > do_sound_norm:False
 | > stats_path:../data/scale_stats.npy
 | > base:10
 | > hop_length:256
 | > win_length:1024

It is different from vocoder_models/en/vctk/hifigan_v2 in mel_fmin/fmax, preemphasis, and stats_path.

Do you have any advice to reduce the artifacts from generated audio?
If the audio parameters are suspicious, then I would train TTS with new parameters.
Anyway I think it is good if models can share same parameters.

[erogol]

Can you also post your model config here?

[erogol]

Ok I'll get the models and make them available with the next release. Thanks again :)

[skol101]

@kaiidams could you share your updated config.json as well as tell how did you manage to continue training the model?

[skol101]

LR is too small. anything smaller than 1e-8 does not really do something. Change the scheduler parameters in the config and let it decay less aggressively. Or use a larger initial lr.

Well, so default parameters should actually be updated in the repo, because 1e-8 is reached after just about 10k-12k steps.
Which LR param should be updated as there 3:
"lr_gen": 0.0001,
"lr_disc": 0.0001,
...
"lr": 0.0001

[kaiidams]

@skol101 In my case, I started from the pretrained English model. The learning rate of the model is already decayed, so the fine-tuning didn't do anything. LR of my configuration is not very useful if you want to try another training. Probably someone who tries to train another model from scratch can figure out better LR.

[skol101]

I see. I'm just observing that HiFiGan training doesn't progress beyond 5k steps: "loss_0" (a target loss function of hifigan) converged to around 19 and not moving anywhere. And I don't understand what I'm doing wrong.

[kaiidams]

#677 (reply in thread) answers the question.