Add one example to run batch inference distributed on Ray

[c21]

This PR is to add an example to run vLLM batch inference in a multi-nodes environment.
Ray Data is used here to orchestrate the workflow:

• Read from multiple input files on cloud storage.
• Start vLLM engines on multiple nodes, run inference on all input in parallel.
• Write the result and prompt back to cloud storage.

Tested w/ 58k number of prompts and meta-llama/Llama-2-7b-chat-hf. Whole job took 5 minutes on 10 L4 GPU nodes.

[zhe-thoughts]

This is a small and clean example. lgtm

cc @simon-mo

[Yard1]

Let's verify this works with tensor parallelism.

[c21]

Let's verify this works with tensor parallelism.

@Yard1 - confirmed tensor parallelism works when testing num_gpus=0 with tensor_parallel_size=2. Underlying 2 child actors (logical GPU=1 per each) got scheduled, and 2 GPUs are actually used. I added comment in PR for tensor parallelism, do not set num_gpus for now.

[c21]

Hi @Yard1 and @simon-mo - addressed all comments, and this PR is ready for review again. Thanks.

[c21]

Thank you @simon-mo and @Yard1 for review!