How to compare results of a new model with yours?

[AlexChungA]

In your results you show a table of the SI-SNRi over the input mixture. In the table it says that for 2spk you obtained 20.1. I have used the code you provide in the repo and the exact commands you provide in the Readme, and when I evaluated the model it gives me as result:
INFO:main:Test set performance: SISNRi=2.22 PESQ=0.0, STOI=0.0.
{"sisnr": 2.2177312467247248, "pesq": 0.0, "stoi": 0.0}.
Which obviously is not the same as the result from the table for 2spk...
I have built a little dataset, similar in number of voices to the one provided with the code, for each number of speakers: 2,3,4 and 5. These are from people speaking in spanish and I want to compare, after training, the performance with the results you got from a dataset of english speakers. How could I do this?

[adiyoss]

Hi @AlexChungA,
Did you train the model on the dataset we provided or on the wsj2-mix?
In case you trained your model on the dataset we provided, it is just a toy dataset (only a few examples from LibriMix) so others can debug and understand the code.
In case you want to reproduce our results you need to first generate the wsj2-mix dataset and optimize a model using this dataset (instructions here: https://enk100.github.io/speaker_separation/).
If you do not have access to the wsj dataset you can train a model using LibriMix dataset (publicly available here: https://github.com/JorisCos/LibriMix), and test it on your own data. Unfortunately, I do not have exact SI-SNRi numbers for LibriMix training (if you will get them, feel free to post them here and I'll update the readme)

[adiyoss]

I will close this issue as there is not new comments for a while.
Feel free to reopen it if needed.