Skip to content

The online-serving gate aborts on current main: harness looks for examples/server, the build produces examples/vllm-server #222

Description

@localai-bot

The canonical online-serving gate cannot run against current main. It aborts
before starting a server.

examples/CMakeLists.txt declares the CMake target server but sets its
output name to vllm-server:

69:  add_executable(server server/main.cpp)
86:  set_target_properties(server PROPERTIES OUTPUT_NAME vllm-server)

The harness checks the on-disk path, which no longer exists:

scripts/dgx-online-serving.sh:293  [[ -x ${build_dir}/examples/server ]] || {
scripts/dgx-online-serving.sh:294    echo "provenance-recorded build did not produce examples/server"
scripts/dgx-online-serving.sh:382,424,693   same path
tools/bench/online_gate.py:3596    "server": build_dir / "examples" / "server"
tools/bench/online_gate.py:3756    server_path = build_dir / "examples" / "server"

Verified on a clean sm_121a Release build of bc6e3d72: examples/ contains
vllm-server, vllm-bench, vllm-cli, and no server.

Introduced by 1a02ab4f feat(release): package canonical server (W6), which set
the release output name without updating the harness that consumes it.

Why this matters beyond a broken path

--execute is how a 27B/35B online-serving ratio becomes an accepted
measurement. While it aborts, throughput numbers for the gate models can only
come from ad-hoc harnesses that do not enforce the checks this one does — the
pinned revision, the oracle version and dependency inventory, cache-drop proof,
one-model/one-lock, and the recorded run recipe. That is the difference between
a number and evidence.

Fix

Point the harness at the built artifact and add a guard so the two cannot drift
again: assert the path the harness expects equals the OUTPUT_NAME declared in
examples/CMakeLists.txt, rather than hardcoding either spelling in two places.

A red test must fail on today's tree before the fix.

Related

  • Qwen3.6 NVFP4 baselines (27B and 35B-A3B) must reach vLLM speed parity #213 (Qwen3.6 NVFP4 speed parity) is blocked on this for any binding number.
  • tools/bench/online_gate.py has model keys 27 -> unsloth/Qwen3.6-27B-NVFP4
    and 35 -> nvidia/Qwen3.6-35B-A3B-NVFP4. There is no key for
    nvidia/Qwen3.6-27B-NVFP4
    , which is the 27B the campaign targets, so that
    model has no reproducible canonical recipe at all. Worth adding in the same
    area of the code.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions