Optimized bad word ids

[guillaume-be]

What does this PR do?

This PR optimizes the generation routines when bad_word_ids are provided by the user. Currently, two inefficiencies significantly slow down the text generation when these are passed:

• single-token bad words are looped over at each generation iteration: this is not required as these single word tokens are always banned, regardless of the input generated so far.
• The token_ids are queried in nested for loops, causing unnecessary and inefficient cross-device communication if these are on the GPU.

The issue was raised in https://discuss.huggingface.co/t/gpt2-many-bad-words-ids-leading-to-slow-text-generation/9721
I could reproduce the issue and observed a severe slowdown of the generation when ~2000 bad word ids are provided (see https://gist.github.com/guillaume-be/2a3e91951869414b6f1f8ab8c2cd642f gist). I observed a ~20x slowdown of the generation when using the bad words with a GPU, from ~1.7s to >25s per generation.

This PR fixes the issue by:

• Moving all of the current token ids to a Python list before multiple iteration through that list, leading to a 20x speedup
• Splitting the bad word ids into 1-element bad words, and words that are made of multiple sub-tokens. For the 1-element bad words, a static bad word pas is pre-computed and re-used for each generation step. This accelerates the generation by a further ~10%.

Fixes https://discuss.huggingface.co/t/gpt2-many-bad-words-ids-leading-to-slow-text-generation/9721

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case: https://discuss.huggingface.co/t/gpt2-many-bad-words-ids-leading-to-slow-text-generation/9721
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@patrickvonplaten - maybe you would like to have a look?

[patrickvonplaten]

Thanks a lot for the improved generation processor here!