Extract confidence

[emanueleielo]

Hi! Actually there is a way to extract the confidence on that segment with the use of avg_logprob or maybe some parameter?
If it is not possible, is in schedule to add a such awesome feature?

[glangford]

See the discussion in

• Confidence scores for each word? #284

[emanueleielo]

Hi!

Actually I saw it but i’m using a cloud whisper so I can’t edit the code. This is the reason I ask to extract the confidence from the avg logprob

[jongwook]

if you take exp() of the avg_logprob, it would somewhat correspond to segment-level confidence. Mathematically the probability/confidence would be the exponential of the sum of the logprobs rather than the average, but practically the number gets too small if you keep multiplying the per-token probabilities, to the point that average logprobs are more useful to quickly check how the model was confident overall for the segment.

Word- or token-level confidence as discussed in #284, #991, or #1119 will be more theoretically grounded, but it's still gets mixed influence from the language modeling perspective (how likely a word would follow another word) and ASR (how likely the speech contained this word).

[emanueleielo]

Thank you! Your answer is totally what I need