GlowTTS - Single vs Multi Speaker - No Phoneme, Common Voice Dataset

[iprovalo]

I am training a GlowTTS model in Ukrainian language using the common voice dataset. Using no phonemes.

I have tried two models - single speaker with about 8K samples, and speaker embedding using the entire set ~70K.

Average loss looks better for the multi speaker model (magenta):

Inference time, the single speaker quality is significantly better than the multi speaker. However, the single speaker still has a slight metallic sound.

I am using multiband melgan for the vocoder. Training steps/times:

1. Single Speaker Glow TTS: 1m steps, 6 days
2. Multi Speaker Glow TTS: 2m steps, 16 days
3. Multiband Melgan: 6m steps, 21 days

Any ideas about why the multi-speaker quality is not matching the single speaker quality?

Thank you!