Evaluation

To run evaluation on anonymized data:

1. Run Stages 0-1 (download dev+test data and evaluation models)

2. Run Stage 8 (prepare evaluation data)

Result: data folders in \baseline\data\

These folders are the Kaldi format data containing: utt2spk, spk2utt, spk2gender, wav.scp and additionally trials and enrolls files in the corresponding directories.

3. Prepare anonymized wav files for:

libri_dev_{enrolls,trials_f,trials_m},

libri_test_{enrolls,trials_f,trials_m},

vctk_dev_{enrolls,trials_f_all,trials_m_all},

vctk_test_{enrolls,trials_f_all,trials_m_all}.

and put the anonymized data for each corresponding folder with name data/$name to the folder with name data/$name_anon. The obtained folders should be in the Kaldi format (i.e, contain utt2spk, spk2utt, spk2gender, wav.scp). Files utt2spk, spk2utt, spk2gender, trials, enrolls should be copied from the corresponding folders with the original data.

*trials_m_all and *trials_f_all - are data corresponding to the joint trial lists for common and different parts of the vctk data sets (they should be anonymized together, but evaluated separately).

4. Run Stages 10-14 (evaluation)

Results and scores for submission:

Result dir with scores and result.txt for submission is created in the directory: ./baseline/exp/results-year-month-day-hours-minutes-seconds/