Skip to content

Qwen3.8-27B (bf16): the token gate and every speed axis are OWED — the checkpoint loads and runs, and has never been adjudicated or measured #915

Description

@localai-bot

Goal

Adjudicate and measure the bf16 arm of Qwen/Qwen3.8-27B @ 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0.
#821 owns the NVFP4 and Q4_K_M
arms of the same checkpoint; neither covers bf16, and the bf16 arm is the one
that already runs.

Why this exists

porting-inventory.md deviation 17
records that this checkpoint "fits GB10 and loads on current main with no code
change", and that "its own token-exact gate is OWED and unrun". Loading is not a
token, and a token is not a throughput number, so two axes were open with no
issue naming them.

Scope

  • A binding greedy token gate against the pinned oracle (555967922), with
    every first-divergence position adjudicated against kNearTieMnats = 500.
  • Throughput / latency / memory against vLLM's production configuration.
  • Records: the owning model-matrix.md row, docs/STATUS.md,
    docs/BENCHMARKS.md under a new key.

Out of scope

The NVFP4 and Q4_K_M arms (#821),
the vision path, and any fix for the tie-break divergence
(#910).

Constraints

dgx.casa (GB10, sm_121a). Every GPU command serialized through $HOME/gpu.lock.
The vLLM arm is never --enforce-eager. Build with
-DVLLM_CPP_CUTLASS_FETCH=ON -DVLLM_CPP_TRITON=ON and the configure-log
fast-path lines asserted, or the number is void.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions