_strip_incomplete_words cuts complete words

[azziko]

Hello,

I saw this and wondered, wouldn't this cut complete words too? Assume, nothing is in the alignatt threshold, the hypothesis is:

['▁U', 'ser', '▁Inter', 'ac', 'tion', '.']

Then the whole Interaction is cut, even though it was not in the frame_threshold

simulstream/simulstream/server/speech_processors/base_streamatt.py

Line 232 in f6688fe

selected_tokens = self._strip_incomplete_words(selected_tokens)

If this is not intended, I would put it like this:

        # Truncate tokens up to the first invalid alignment (if any)
        if len(invalid_tok_ids) > 0:
            selected_tokens = selected_tokens[:invalid_tok_ids[0]]
            if self.word_level_postprocess:
                selected_tokens = self._strip_incomplete_words(selected_tokens)

But maybe that's intended for the models that output partials words

[sarapapi]

Hi, thanks for pointing it out. I'll make a PR to correct this.

[mgaido91]

I am not sure this is a bug to be honest. This is rather a design choice. It also depends on how BPEs are handled. In the past, when BPE would put a @@ after tokens that have a continuation (i.e. not followed by a space), rather than a _ in front of the leading BPE, you could know earlier when a word is over.

What we can do here is rely also on the punctuation, but I would consider as a different variant of the current method. We can do that, but I would not remove the previous behavior. Also, @azziko feel free to open PRs if you prefer to have your contributions acknoledged and be listed in the contributors of the repo.