GLM-5.3-Flash on 8x B200: dual TP4/EP4 serving profile with long-context measurements

[stewtong]

I measured an SGLang serving profile for zai-org/GLM-5.3-Flash on one 8x NVIDIA B200 node. It uses two independent TP4/EP4 replicas, FP8 KV cache, TRT-LLM DSA backends, the FlashInfer TRT-LLM MoE runner, static NEXTN 5-1-6, and session-affine routing.

The pinned software stack, engine commands, proxy configuration, client setup, methods, and results are documented here: https://github.com/stewtong/b200-glm53

Validation observations:

• All 138 streaming Chat Completions requests returned HTTP 200 with the requested completion length. None timed out or returned HTTP 429.
• At concurrency 1, median end-to-end output rate including prefill was 245.3 tok/s at about 131K input tokens and 12.50 tok/s at about 1M input tokens.
• At about 524K input tokens and total concurrency 16 across both replicas, 48/48 requests completed without an observed out-of-memory error. Maximum capacity remains unmeasured.
• GSM8K scored 1,284/1,319 (97.35%) with the checkpoint's default max effort and 1,272/1,319 (96.44%) with reasoning_effort: low. The conditions used different output limits and are reported separately in the repository.
• In a repeated-prefix cache study with stable session affinity, median time to first output event was 3.10 seconds on turn 1 and 0.62 seconds on later turns. With a changed leading system line, the corresponding medians were 3.13 and 3.24 seconds.

The serving table and cache study used three repetitions per condition. The custom harness, prompt corpus, and raw traces are unpublished, so exact numerical reproduction requires an equivalent harness and prompt set. The repository documents the tested boundaries and excludes unsupported aggregate throughput, decode TPOT, route-latency, and causal claims.