[Question]SpeakerNet confidence in each label

[JINHXu]

Hey there,

I was wondering if there is a way to get the confidence/probability in each label in the SpeakerNet model (in Speaker Identification)? For example, if I have three speakers/labels in fine-tuning data, one audio for inference, how can I obtain the probability of the speaker beinglabel1, label2, and label3? Thanks in advance for your help!

Cheers!
Xu

[nithinraok]

Hi,

We take top1 value here as argmax to get highest probablity label,

NeMo/examples/speaker_recognition/speaker_reco_infer.py

Line 122 in e070e04

infer_labels = all_logits.argmax(axis=1)

. Instead if you would like to get the probability as well as indices of those labels, you may replace it with torch.topk https://pytorch.org/docs/stable/generated/torch.topk.html and handle outputs of topk function accordingly.

[JINHXu]

Thank you!! @nithinraok

[JINHXu]

Hi @nithinraok

Sorry to bother you again with another question... I was wondering what each "logit" stands for/how was it calculated? I think I understand that they reflect the confidence/probability of each label and we would definitely go with argmax as the inference, the bigger the logit the more likely the label it is, but what exactly are those logits? I was wondering if there is more information I can get out of the logits, for example since the current system always return an inference even if it is a speaker not in the training data, it will return the most similar one. However, I was hoping to identify "unknown users"(those who are not in the training data) too. One of my solutions is to add a unk class to the training data, the unk data should be training samples from some random other speakers. The other way I am thinking is to get some clue from the "logits", for example, if the max logit is still lower than some value (a bar), it should be classified as "unknown user" instead of the most similar label (argmax).

(I could not seem to find a clue what each logit is from your paper either:

The intermediate features, S are passed through the Decoder consisting of two layers each of output size 512 for a linear transformation from S to the final number of classes N for the larger (L) model, and a single linear layer of out-put size 256 to the final number of classes N for the medium (M) model. We extract q-vectors after the final linear layer of fixed size 512, 256 for SpeakerNet-L and SpeakerNet-M models respectively.

)

Thank you again for your help!!