[Feature] Code implementation of Async Scheduler

[cychiuak]

feat(scheduler): Implement asynchronous scheduler to reduce CPU wait overhead

Problem:
The existing synchronous scheduler blocks the CPU while waiting for TPU
results. This wait time creates significant overhead, especially for
small models or decode-heavy scenarios where the scheduler is invoked
frequently.

Solution:
This commit introduces an asynchronous scheduler that decouples the CPU
scheduler from the TPU execution. The CPU no longer blocks and can
continue processing, effectively eliminating the wait-time overhead.

Impact:
Observed a ~33% throughput increase on Llama3.2-1B. This change is
most significant for workloads with high scheduler call frequency.

Benchmark (Llama3.2-1B):

Before: 6.38 req/s (Sync @ cychiuak@8c7e7bb)
After: 8.89 req/s (Async)
How to Reproduce:

HF_TOKEN=<YOUR_HF_TOKEN> VLLM_SERVER_DEV_MODE=1 VLLM_USE_V1=1 \
VLLM_TORCH_PROFILER_DIR=/dev/shm/vllm-prof \
python -m vllm.entrypoints.cli.main serve meta-llama/Llama-3.2-1B \
--max-model-len 800 --max-num-seqs 256 --max-num-batched-tokens 512 \
--no-enable-prefix-caching --tensor-parallel-size 1 --async-scheduling

python -m vllm.entrypoints.cli.main bench serve --backend vllm \
--model meta-llama/Llama-3.2-1B --dataset-name random --random-input-len 230 \
--random-output-len 120 --num-prompts 200 --max-concurrency 3 \
--percentile-metrics "ttft,tpot,itl,e2el" --ignore-eos

[py4]

Tnx for the PR. Please add e2e CI test for this feature to prevent feature from breaking

Accidental

[bvrockwell]

Great contribution @yuyanpeng-google !! Excited to see it in action.

[vipannalla]

LGTM, @yuyanpeng-google can follow up on adding the tests to CI after PR is merged.