Possible decoding error in RNNT beam search

[w4-jihunlorenzopark]

Hi, I have found a possible decoding error in current beam search decoding logic: nemo/collections/asr/parts/submodules/rnnt_beam_decoding.py#L663.

I am using default_beam_search of BeamRNNTInfer with partial_hypothesis.
We put the the last token from the previous step's hypothesis in this part. The last token is used to get the probability of predicted token of the current step from the decoder. For instance, If the last token is s, then the result would be {'s?': -2.3274096590466797, 's.': -0.9459273740649223}. Then we remove the last token from the hypothesis of current step and the final result of default_beam_search would be {'?': -2.3274096590466797, '.': -0.9459273740649223} by this logic.

However, in my case, my model somehow results the blank token, for instance, {'s?': -2.3274096590466797, 's': -0.9459273740649223} then, the logic do not filter s, which is the last token of previous hypothesis. It makes the function returns hypothesis like this {'?': -2.3274096590466797, 's': -0.9459273740649223}, which leads the last token 's'` appear once again. (Actually, the original problem here is the model predicts the blank token. Would be thankful if you let me know why this is happening!)

So, my suggestion is how about modifying like this?

        # Remove the original input label if partial hypothesis was provided
        if partial_hypotheses is not None:
            for hyp in kept_hyps:
                if hyp.y_sequence[0] == partial_hypotheses.y_sequence[-1]:
                    hyp.y_sequence = hyp.y_sequence[1:]

                    if len(hyp.y_sequence) == 0:
                        hyp.y_sequence = [self.blank]


        return self.sort_nbest(kept_hyps)

I think the right place is the Issues tab, but I cannot reveal the codes for reproduction for now. I just want to make a quick report without preparing the another codes to reproduce this situation.