HuBERT ExtraLarge instead of HuBert base model?

[HSultankhan]

Not an Issue:
Just wanted to know that Can we use HuBERT Extra Large model instead of HuBERT base model?

Will it perform better and generate better quality results?

Reference : https://github.com/facebookresearch/fairseq/tree/main/examples/hubert

[blakejrobinson]

I did experiment with this. One obvious thing is that you won't be able to run models other people make (unless they also use hubert extra large) as inference requires the same HuBERT that was used to train.

Not sure I could hear much quality difference between trained models, but I didn't go into much detail once I'd realised the above.

[Tps-F]

Support for other HuBerts, including ExtraLarge, is planned.

[HSultankhan]

@Tps-F Looking forward to it,

[Mixomo]

Looking forward! 👀

[thepowerfuldeez]

Hi! Adding some comment here, correct me if I'm wrong. Modern SVC systems use ContentVec instead of plain Hubert, which is special pre-training of Hubert that disentangles speaker features making it better suited for voice conversion task.
Using just larger Hubert might not make better quality out of the end-to-end system.
Instead, combination of Whisper-large-v3 and ContentVec gives better results, as described in this paper: https://arxiv.org/abs/2310.11160

[HSultankhan]

@thepowerfuldeez Have you tried this integration?