b200 kimi k2.5 int4 vllm single node

[functionstackx]

[functionstackx]

@claude implement & create/open/submit an PR for vllm kimi k2.5 int4 tp8 conc from 4 to 64

moonshotai/Kimi-K2.5 --tensor-parallel-size 8 --mm-encoder-tp-mode data

use vllm 0.15.1 image tag

this is following https://github.com/vllm-project/recipes/blob/main/moonshotai/Kimi-K2.5.md

[Klaud-Cold]

Claude finished @functionstackx's task in 5m 9s —— View job • claude/issue-727-20260218-0136 • Create PR ➔

---

Kimi K2.5 INT4 vLLM Benchmark for B200

• Gather context: read existing configs and reference recipe
• Understand config patterns for vLLM single-node benchmarks
• Create benchmark script (benchmarks/kimik2.5_int4_b200.sh)
• Add kimik2.5-int4-b200-vllm config to nvidia-master.yaml
• Update perf-changelog.yaml
• Commit and push changes
• Create PR

Summary

Added Kimi K2.5 INT4 vLLM benchmark configuration for B200 single-node (8xGPU):

New files:

• benchmarks/kimik2.5_int4_b200.sh — vLLM serve script with --mm-encoder-tp-mode data and --trust-remote-code per the vLLM recipe

Modified files:

• .github/configs/nvidia-master.yaml — Added kimik2.5-int4-b200-vllm config entry
• perf-changelog.yaml — Added changelog entry

Config details:

• Model: moonshotai/Kimi-K2.5
• Image: vllm/vllm-openai:v0.15.1
• TP=8, concurrency 4–64
• Sequence lengths: 1k1k, 1k8k, 8k1k

|

[functionstackx]

dont do no prefix cache & max-model-len crap, delete it @claude please

# Calculate max-model-len based on ISL and OSL
if [ "$ISL" = "1024" ] && [ "$OSL" = "1024" ]; then
    CALCULATED_MAX_MODEL_LEN=$((ISL + OSL + 20))
elif [ "$ISL" = "8192" ] || [ "$OSL" = "8192" ]; then
    CALCULATED_MAX_MODEL_LEN=$((ISL + OSL + 200))
else
    CALCULATED_MAX_MODEL_LEN=${MAX_MODEL_LEN:-10240}
fi

cat > config.yaml << EOF
no-enable-prefix-caching: true
max-model-len: $CALCULATED_MAX_MODEL_LEN
EOF

[Klaud-Cold]

Claude finished @functionstackx's task in 1m 38s —— View job • claude/issue-727-20260218-0159

---

Remove prefix cache & max-model-len config from Kimi K2.5 script

• Read current benchmark script
• Remove no-enable-prefix-caching and max-model-len calculation block
• Commit and push changes

Removed from benchmarks/kimik2.5_int4_b200.sh:

• The max-model-len calculation block (ISL/OSL-based CALCULATED_MAX_MODEL_LEN)
• The config.yaml generation with no-enable-prefix-caching: true and max-model-len
• The --config config.yaml flag from vllm serve
• MAX_MODEL_LEN from check_env_vars

vLLM will now use its default prefix caching and model length behavior.

Create PR

View job | claude/issue-727-20260218-0159

[functionstackx]

@claude rebase such that it branches off of branch claude/issue-723-20260218-0123 and b200 uses --trust-remote-code on benchmark script too

[Klaud-Cold]

Claude finished @functionstackx's task in 3m 17s —— View job • claude/issue-727-20260218-0415 • [Create PR ➔](https://github.com/SemiAnalysisAI/InferenceX/compare/main...claude/issue-727-20260218-0415?quick_pull=1&title=Issue%20%23727%3A%20Changes%20from%20Claude&body=This%20PR%20addresses%20issue%20%23727%0A%0AGenerated%20with%20%5BClaude%20Code%5D(https%3A%2F%2Fclaude.ai%2Fcode)

---

Rebase onto claude/issue-723-20260218-0123 and add --trust-remote-code

• Gather context: check current branch state and source branch
• Read existing Kimi K2.5 benchmark script from previous branch
• Reset branch to be based on claude/issue-723-20260218-0123
• Create B200 benchmark script with --trust-remote-code on both vllm serve and run_benchmark_serving
• Update nvidia-master.yaml with kimik2.5-int4-b200-vllm config entry
• Update perf-changelog.yaml
• Commit and push changes

Summary

Rebased claude/issue-727-20260218-0415 to branch off claude/issue-723-20260218-0123 (which has the MI355X Kimi K2.5 work + --trust-remote-code passthrough in benchmark_lib.sh).

Commit history now:

eebc476 Add Kimi K2.5 INT4 vLLM benchmark for B200
9428947 Add --trust-remote-code passthrough to benchmark_lib.sh
d7ce514 Remove --max-seq-len-to-capture flag from Kimi-K2.5 benchmark
e4bf03f Update perf-changelog PR link to #734
3975c4d Add Kimi-K2.5 INT4 vLLM benchmark for MI355X
a68a870 update code owner (#711)

Changes in the B200 commit:

• benchmarks/kimik2.5_int4_b200.sh — --trust-remote-code on both vllm serve and run_benchmark_serving
• .github/configs/nvidia-master.yaml — Added kimik2.5-int4-b200-vllm config (TP=8, conc 4-64, image vllm/vllm-openai:v0.15.1)
• perf-changelog.yaml — Added changelog entry

Create PR

View job | claude/issue-727-20260218-0415