HuggingFace ECAPA model - What data were used for training?

[fra1993]

Hi, did you use an external RIR dataset to train this model as it happens by default when retraining with the standard ECAPA receipe in SpeechBrain? Thanks.

[anautsch]

Hi @fra1993 to which of the ECAPA pretrained models are you referring to?

Please take a look through our recipes:
https://github.com/speechbrain/speechbrain/tree/develop/recipes

They are sorted by databse, then by tasks in nested folders. Usually, we put on HuggingFace only pretrained models from our recipes.

For example, the VoxCeleb one refers to:

speechbrain/recipes/VoxCeleb/SpeakerRec/hparams/train_ecapa_tdnn.yaml

Lines 91 to 99 in c723843

	add_rev: !new:speechbrain.lobes.augment.EnvCorrupt
	openrir_folder: !ref <rir_folder>
	openrir_max_noise_len: 3.0 # seconds
	reverb_prob: 1.0
	noise_prob: 0.0
	noise_snr_low: 0
	noise_snr_high: 15
	rir_scale_factor: 1.0

So there's a hint to speechbrain.lobes.augment:

speechbrain/speechbrain/lobes/augment.py

Line 29 in c723843

OPENRIR_URL = "http://www.openslr.org/resources/28/rirs_noises.zip"

Hope this helps to read SpeechBrain code better.

[fra1993]

Hi, thanks for the answer. I was referring to speechbrain/spkrec-ecapa-voxceleb: the pretrained model for speaker verificaton with speaker embedding dimension = 192. I believe that this model uses an external RIR database for augmentation but I want to be 100% sure.

[anautsch]

@fra1993 I don't know how to reply to this better: the link is posted above.

Did you try debugging your Python application?
e.g. https://www.jetbrains.com/help/pycharm/debugging-your-first-python-application.html

Above I demonstrated to you how to do it manually, and since that is not a working method for you, I'd encourage you to use the debugger. Again, please check the links above, for it might outline already what you are asking for. For putting a debug point—as already suggested indirectly, but now more directly—please take a look and debug this function within the recipe you are curious about:

speechbrain/speechbrain/lobes/augment.py

Line 302 in 055d465

class EnvCorrupt(torch.nn.Module):

To identify which recipe you are curious about...

1. here are all our recipes (basis for pretrained models) -> https://github.com/speechbrain/speechbrain/tree/develop/recipes
2. VoxCeleb -> https://github.com/speechbrain/speechbrain/tree/develop/recipes/VoxCeleb
3. hparams to check emb dim == 192 -> https://github.com/speechbrain/speechbrain/tree/develop/recipes/VoxCeleb/SpeakerRec/hparams
4. this folder has a train_ecapa_tdnn.yaml (same file as linked above)

speechbrain/recipes/VoxCeleb/SpeakerRec/hparams/train_ecapa_tdnn.yaml

Lines 66 to 78 in 055d465

	embedding_model: !new:speechbrain.lobes.models.ECAPA_TDNN.ECAPA_TDNN
	input_size: !ref <n_mels>
	channels: [1024, 1024, 1024, 1024, 3072]
	kernel_sizes: [5, 3, 3, 3, 1]
	dilations: [1, 2, 3, 4, 1]
	groups: [1, 1, 1, 1, 1]
	attention_channels: 128
	lin_neurons: 192
	classifier: !new:speechbrain.lobes.models.ECAPA_TDNN.Classifier
	input_size: 192
	out_neurons: !ref <out_n_neurons>

I hope this how-to guide can also serve as a reference point for others with similar question of how to link SpeechBrain pretrained models on HuggingFace with SpeechBrain recipes to figure out some detail in the training procedure.

[fra1993]

Hi, yes I checked all these things when training my network. I guess I just wanted to be sure you used the RIR augmentation for the pretrained model you loaded on huggingface which looked like the case from the yaml file in the recipes folder but I needed to be sure. Thanks

[anautsch]

I did not write the recipe, I did only perform a quick check ... other recipes use additional noise sources. Please check which files are created on your local disk when running this recipe. Dummy data is enough, e.g., as of our recipe tests
https://github.com/speechbrain/speechbrain/tree/develop/tests/recipes

All recipes of the VoxCeleb dataset can be checked using this command:

python -c 'from tests.utils.recipe_tests import run_recipe_tests; print("TEST FAILED!") if not(run_recipe_tests(filters_fields=["Dataset"], filters=[["VoxCeleb"]], do_checks=False, run_opts="--device=cuda")) else print("TEST PASSED")'

@mravanelli did a great job last summer in outlining the first steps for such testing.

It's a few weeks ago since I worked on that closely, but it should download all the noises too which are necessary for mixing data etc.—and if not, depending overrides can be found under test_debug_flags here:
https://github.com/speechbrain/speechbrain/blob/develop/tests/recipes/VoxCeleb.csv

I understand the desire to be 100% — been there.

If you like to use SpeechBrain more frequent, take a pause to ponder, how the building blocks from yaml & train recipes play together. You'll need some fundament to stand on. Such, one can only build for oneself.

Look, if you know where you are stuck—like at which point which method is lacking—, just ask about it. It will help you and others.