Skip to content

Commit

Permalink
whisper : make beam candidate sort more stable (#1943)
Browse files Browse the repository at this point in the history
All else being otherwise equal, this encourages the beam candidate
selection to re-use the same decoder, which slightly
reduces the cache size.

I wouldn't expect it to make much of a performance difference,
but it helps when debug printing the cache and beam.

Added as part of understanding #1941.
  • Loading branch information
josharian authored Mar 9, 2024
1 parent ce945b5 commit 2852e1a
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion whisper.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5357,7 +5357,10 @@ int whisper_full_with_state(
beam_candidates.begin(),
beam_candidates.end(),
[](const beam_candidate & a, const beam_candidate & b) {
return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
if (a.sequence.sum_logprobs_all != b.sequence.sum_logprobs_all) {
return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
}
return a.decoder_idx < b.decoder_idx;
});

uint32_t cur_c = 0;
Expand Down

0 comments on commit 2852e1a

Please sign in to comment.