[Continous Batching] fix do_Sample=True in continuous batching

[kashif]

This pull request introduces improvements to sampling support in continuous batching generation, enhances tensor shape handling for logit processors, and expands test coverage for batch generation with sampling. The main changes are grouped below:

Sampling and Generation Behavior:

• Enabled sampling in batch_generate by setting do_sample=True and configuring temperature and top_p parameters, allowing for more diverse outputs during batch generation.

Tensor Shape Handling and Sampling Logic:

• Updated the logit processor in continuous_api.py to handle 3D logits produced by continuous batching, reshaping them as needed for compatibility with logit processors and restoring original shapes after processing.
• Improved sampling logic in _sample to ensure correct tensor dimensions when sampling tokens, making the method robust for both sampling and greedy decoding.

Paged Attention and Integration:

• Added a check for the presence of flash_attn_varlen_func in the attention implementation to ensure compatibility and prevent errors during paged attention forward passes.

Testing and Validation:

• Refactored batch output decoding in tests to use generated_tokens instead of static_outputs for accuracy.
• Added a new parameterized test test_generate_batch_with_sampling to verify that batch generation with sampling enabled works correctly across different attention implementations and configurations. The test checks for successful completion, non-empty outputs, and actual sampling behavior.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[McPatate]

lgtm, ty!

[ArthurZucker]

thanks! perfect 🤗