Add multinomial sampling to sequence generation

[seanmor5]

Adds multinomial sampling strategy and top-k/top-p options for adjusting the distribution.

[jonatanklosko]

Since this function is pretty higher-level, what about :seed instead?

[seanmor5]

Agreed, though one thing I was thinking was if we should pass this in the serving as well and return the updated key. They do accept key in generate in HF, but they do not return an updated key, so maybe just allowing a key to the serving is enough and it is assumed you will split it outside the serving

[jonatanklosko]

Servings are so high-level that it seems to me the seed is for reproducibility if anything else (or to explore different result for the same prompt), and so a number is more transparent than a tensor key. Or do you think there's a scenario where you would actually use a split key?

[seanmor5]

Sorry, what I mean is currently you'd need to create a new serving with a new seed to get different inputs. Maybe this is desirable though, but I'm not sure as if you want something non-deterministic you may want the option to pass a key to the serving

[jonatanklosko]

@seanmor5 I finished sampling, we can do beam search separately, especially that we have contrastive search already :)

[josevalim]

we can do beam search separately

If you think about it, every search we do is a beam search.