Question: The details of Pretraining

[YutoNishimura-v2]

Hello,

Thank you for providing such great code. I would like to know more about the pretraining process to better utilize this code. Specifically, I would like to know the following information:

・The dataset used for pretraining
・Techniques used during training

For the second point, I would like information such as "gradually increasing the number of speakers" that is necessary for reproducing the training.

Thank you.

[RVC-Boss]

dataset： VCTK
Techniques used during training：The training code is contained in this repo. (training button) But you should prepare the data and filelist.txt(wav_path,npy_path,speaker id) before training. The detail is in train_nsf_sim_cache_sid_load_pretrain.py.
Multi speaker is supported for training. WebUI doesn't support it now.
I did not carefully count the number of epochs required for pretraining, but I trained per single model using 4 V100 for 3 days.

[YutoNishimura-v2]

Thank you for your response!

To be honest, I am shocked that the pretraining of this amazing quality model was only done using the VCTK dataset. Why did you choose not to use more English and Japanese data to train the model? I can imagine that the quality would be even better if you had used more data.

[tarepan]

Can I ask you an additional question?
What batch_size is used for pretraining?

In config file for fine-tuning, it is set to 4.
It seems to be small for 4xV100 setup for pretraining.

[RVC-Boss]

Can I ask you an additional question? What batch_size is used for pretraining?

In config file for fine-tuning, it is set to 4. It seems to be small for 4xV100 setup for pretraining.

bs16=4*4（GPU memory:11G/16G）

[YutoNishimura-v2]

Thank you for your response!

To be honest, I am shocked that the pretraining of this amazing quality model was only done using the VCTK dataset. Why did you choose not to use more English and Japanese data to train the model? I can imagine that the quality would be even better if you had used more data.

Sorry, how about this future work?

[RVC-Boss]

Thank you for your response!
To be honest, I am shocked that the pretraining of this amazing quality model was only done using the VCTK dataset. Why did you choose not to use more English and Japanese data to train the model? I can imagine that the quality would be even better if you had used more data.

Sorry, how about this future work?

@YutoNishimura-v2
More Chinese Singing training-set for pretraining model.
(fix: More data(including Chinese Singing training-set))
Less artifact on breath.
Better retrieval faiss index.
F0 editor.
Better f0 extract pipeline.
More discriminators for pretraining model.

[YutoNishimura-v2]

@RVC-Boss
Thank you!

Sorry,

More Chinese Singing training-set for pretraining model.

you said:

dataset： VCTK

"More" means that you already use some chinese singing dataset for pretraining?

[RVC-Boss]

No, now VCTK, more data will be in the future.

[YutoNishimura-v2]

Understood. Thank you!!

[BoltzmannEntropy]

Which script is used for training a multi-speaker model?

[kelseyjd]

@YutoNishimura-v2 More Chinese Singing training-set for pretraining model. (fix: More data(including Chinese Singing training-set)) Less artifact on breath. Better retrieval faiss index. F0 editor. Better f0 extract pipeline. More discriminators for pretraining model.

@RVC-Boss curious if you have tried any new discriminators since this last comment. I was thinking to try a multiresolution spectral discriminator such as the one used in UnivNet to address high-frequency artifacting. Did you try any new ones yourself?