[tilert] Add glm5.1-fp8-b200-tilert 1k1k+8k1k: vLLM prefill + TileRT decode PD-disaggregation / 新增 glm5.1-fp8-b200-tilert 1k1k+8k1k：vLLM prefill + TileRT decode PD 分离

[CrimsonDump]

Summary

Adds a new framework tilert for GLM-5.1 FP8 on 8×B200, running prefill/decode disaggregated:
stock vLLM prefill (TileRTConnector as a kv_producer) hands KV over NIXL to a TileRT
decode_server, and tilert's pd_router exposes an OpenAI-compatible endpoint.

No vLLM fork or patch — the connector loads through vLLM's standard kv_connector_module_path.

TileRT decode serves bs=1 only, so this contributes a single point at conc=1 rather than a
Pareto curve. This is stated explicitly in the config and the changelog entry.

Results (validated on our own 2× 8×B200)

Scenario	Metric	Value
8k/1k, conc=1	interactivity	313.6 tok/s/user (mean TPOT 3.19 ms)
gsm8k (full 1319, official eval config)	strict-match	0.9773
	flexible-extract	0.9757

Threshold is 0.93; both pass. CI runs will of course have to reproduce this — these numbers are
from our own hardware and are provided for context only.

What this PR adds

File	Purpose
configs/nvidia-master.yaml	the glm5.1-fp8-b200-tilert entry (multinode, disagg, kv-p2p-transfer: nixl, prefill/decode TP8, conc-list: [1], ISL 1k and 8k)
perf-changelog.yaml	one appended entry
runners/launch_b200-dgxc.sh	a tilert branch that injects dgxc-specific settings and execs the thin entry point
benchmarks/multi_node/glm5.1_fp8_b200_tilert-disagg.sh	thin entry point carrying model/precision/parser settings
benchmarks/multi_node/tilert_utils/{submit,run_node,setup_deps}.sh	orchestration, modelled on amd_utils/

584 insertions, 0 deletions — no existing line is modified anywhere in the repo. The launcher
branch is a pure insertion that execs before any dynamo code runs, so every dynamo path is
byte-for-byte unchanged.

Prefill and decode need different container images because tilert pins
transformers==4.46.3 while vLLM needs >=5.5.3 — they cannot share one environment. This
happens to match the P/D physical topology.

MTP is mandatory here rather than an optional speedup: the PD wire format carries 79 KV layers
(78 main + 1 MTP draft), so the handshake rejects a non-MTP prefill.

Checklist note — "must not patch the inference engine or serving stack"

Flagging this proactively. tilert_utils/setup_deps.sh runs pip install tilert==0.1.5.post2 at
container start. We believe this is not a patch:

• The official tilert image ships no tilert package — it is a bare base environment, and
  runtime installation is the standard step in TileRT's own README.
• On the prefill side we install with --no-deps, so only the connector plugin lands. vLLM
  itself is untouched — no fork, no rebuilt engine wheel, no site-packages edits, no
  monkey-patching. The connector is loaded through vLLM's documented
  kv_connector_module_path extension point.
• The decode side additionally upgrades transformers>=5.4, which the official checkpoint's
  TokenizersBackend requires. That side runs tilert, not a vLLM/SGLang engine.
• In-tree precedent: benchmarks/multi_node/amd_utils/setup_deps.sh installs dependencies at
  container start the same way.

If reviewers still consider this in scope, we are happy to file a waiver under
docs/waiver/<PR_NUMBER>.md.

Open questions (blocking — we would like guidance before requesting review)

1. GLM-5.1 is currently retired. MODELS.md:143 records GLM-5/GLM-5.1 as retired on
2026-07-18 (#2276), and the review checklist requires verifying that a PR does not submit a
deprecated model. Should glm5.1 be restored to the active list first (a separate PR?), or should
this submission take the "additional reasoning" exception path?

2. CODEOWNER ownership. .github/CODEOWNERS assigns configs/nvidia-master.yaml to
@ankur-singh @kedarpotdar-nv @InferenceX/core. Our entry lands in that file because the
benchmark runs on B200, but we are the TileRT team, not NVIDIA. Should the current owners
sign off, or would you prefer a CODEOWNERS rule scoping the tilert paths to our team (mirroring
experimental/operatorx/ @hbarclay)?

The dgxc-infrastructure questions are tracked separately in #2524 — weight path, ~700 GiB
storage for the converted 8-shard (and whether you could pre-stage it), lustre flock, RDMA
capabilities under pyxis, UCX NIC pinning, all-evals for a conc=1 entry, and b200-multinode
runner access. That issue also carries a measured timing table for planning.

---

概述

新增 framework tilert，在 8×B200 上以 PD 分离方式跑 GLM-5.1 FP8：原版 vLLM 负责 prefill
（TileRTConnector 作为 kv_producer），KV 经 NIXL 交给 TileRT decode_server，由 tilert 的
pd_router 对外提供 OpenAI 兼容端点。

不 fork 也不 patch vLLM —— connector 通过 vLLM 标准的 kv_connector_module_path 加载。

TileRT 解码只支持 bs=1，因此本条目只贡献 conc=1 的单个点，画不出 Pareto 曲线。这一点在
config 和 changelog 条目里都已明确写出。

实测结果（我们自有的 2× 8×B200）

场景	指标	数值
8k/1k，conc=1	interactivity	313.6 tok/s/user（平均 TPOT 3.19 ms）
gsm8k（全量 1319 题，官方 eval 配置）	strict-match	0.9773
	flexible-extract	0.9757

阈值 0.93，双双达标。CI 当然需要重新复现 —— 上述数字来自我们自己的机器，仅供参考。

本 PR 的内容

文件	职责
configs/nvidia-master.yaml	glm5.1-fp8-b200-tilert 条目（multinode、disagg、kv-p2p-transfer: nixl、prefill/decode 均 TP8、conc-list: [1]、ISL 1k 与 8k）
perf-changelog.yaml	末尾追加一条
runners/launch_b200-dgxc.sh	tilert 分支，注入 dgxc 专属配置后 exec 到薄入口
benchmarks/multi_node/glm5.1_fp8_b200_tilert-disagg.sh	薄入口，承载模型/精度/parser 配置
benchmarks/multi_node/tilert_utils/{submit,run_node,setup_deps}.sh	编排层，对标 amd_utils/

584 行新增、0 行删除 —— 全仓没有修改任何既有行。launcher 分支是纯插入，且在所有 dynamo 代码
之前 exec 走，因此每一条 dynamo 路径都逐字节未变。

Prefill 与 decode 必须用不同镜像，因为 tilert pin transformers==4.46.3 而 vLLM 需要
>=5.5.3，装不进同一环境。这恰好与 P/D 的物理拓扑吻合。

MTP 在这里是必需项而非可选加速：PD 线格式携带 79 个 KV 层（78 主 + 1 MTP draft），握手会拒绝
不开 MTP 的 prefill。

Checklist 说明 —— 关于「不得 patch 推理引擎或服务栈」

主动说明。tilert_utils/setup_deps.sh 在容器启动时执行 pip install tilert==0.1.5.post2，
我们认为这不构成 patch：

• 官方 tilert 镜像本身不含 tilert 包 —— 它是纯基础环境，运行时安装是 TileRT 官方 README 的
  标准步骤。
• prefill 侧用 --no-deps 安装，只落下 connector 插件。vLLM 本体未被触碰 —— 没有 fork、没有
  重编译的 engine wheel、没有改 site-packages、没有 monkey-patch。connector 通过 vLLM 文档化的
  kv_connector_module_path 扩展点加载。
• decode 侧额外升级 transformers>=5.4，这是官方 checkpoint 的 TokenizersBackend 所需。该侧跑的
  是 tilert，不是 vLLM/SGLang 引擎。
• 在树先例：benchmarks/multi_node/amd_utils/setup_deps.sh 同样在容器启动时安装依赖。

如果 reviewer 仍认为这在条款范围内，我们乐意在 docs/waiver/<PR_NUMBER>.md 提交豁免说明。

待确认的问题（阻塞项 —— 希望在请求正式评审前得到指引）

1. GLM-5.1 目前是 retired 状态。 MODELS.md:143 记录 GLM-5/GLM-5.1 于 2026-07-18 退役
（#2276），而评审 checklist 要求确认 PR 未提交已弃用的模型。是应该先把 glm5.1 恢复到活跃
列表（单独一个 PR？），还是本次提交走「额外说明」的例外路径？

2. CODEOWNER 归属。 .github/CODEOWNERS 把 configs/nvidia-master.yaml 指派给
@ankur-singh @kedarpotdar-nv @InferenceX/core。我们的条目因为跑在 B200 而落入该文件，但我们是
TileRT 团队而非 NVIDIA。应该由现任 owner 签核，还是你们更希望新增一条 CODEOWNERS 规则、把 tilert
相关路径划给我们团队（参照 experimental/operatorx/ @hbarclay）？

dgxc 基础设施方面的问题已单独记录在 #2524 —— 权重路径、转换后 8-shard 的约 700 GiB 存放
（以及你们能否直接预置）、lustre 的 flock、pyxis 下的 RDMA 能力、UCX 网卡绑定、conc=1 条目的
all-evals，以及 b200-multinode runner 资源。该 issue 还附了一份实测时间表供排期参考。

---

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[github-actions]

Dispatched trusted sweep run for approved external revision 873f6615ba0b8451f9e8a5ea0fc8f9f5927fd241.
New commits are not trusted automatically; remove and re-add the primary sweep label to approve a new SHA.

已为获批的外部提交 873f6615ba0b8451f9e8a5ea0fc8f9f5927fd241 调度trusted sweep run。
后续新提交不会自动获得信任；如需批准新的 SHA，请移除并重新添加主扫描标签。

[github-actions]

Dispatched trusted sweep run for approved external revision d5f238691fc0d2c065f8091ca1b4f6ea7d8abe81.
New commits are not trusted automatically; remove and re-add the primary sweep label to approve a new SHA.

已为获批的外部提交 d5f238691fc0d2c065f8091ca1b4f6ea7d8abe81 调度trusted sweep run。
后续新提交不会自动获得信任；如需批准新的 SHA，请移除并重新添加主扫描标签。