Skip to content

vLLM oracle version check is unsatisfiable at the current pin: the only build that matches cannot run, and the only build that runs is refused #2896

Description

@localai-org-maint-bot

Row: BENCH-VLLM-ONLINE-SERVING

The bind

The only vLLM build mode that satisfies the harness's version check cannot execute a kernel, and the only mode that executes is refused by that check. No cross-engine speed number can be taken at the current pin on dgx:gpu0 until this is resolved.

The recorded pin (.agents/upstream-sync.md:81):

vllm_distribution_version = 0.28.1rc1.dev132+ge126687a9.precompiled

The check (tools/bench/online_gate.py:3531-3534), an equality, before the commit assertion:

if (
    metadata_version != VLLM_DISTRIBUTION_VERSION
    or runtime_version != VLLM_ORACLE_VERSION
):
    raise HarnessError("vLLM oracle version drift: ...")

VLLM_DISTRIBUTION_VERSION is read straight from the pin (tools/bench/serve_low_common.py:108).

The two arms:

Build mode Distribution version Can it run?
source build no .precompiled suffix → REFUSED yes
VLLM_USE_PRECOMPILED=1 on aarch64 matches → accepted no — produces a 13,872-byte editable wheel with no compiled extensions (.agents/sync/2026-09-02-e126687.md §5.5)

#2818 anticipated the refusal. What it does not record is that the string it is matching against came from an install that cannot execute a kernel on this architecture.

Compounding: no runnable oracle on the gate device at all

The only wheel at e126687a9a (/mnt/nas_share/rc/runhalf-e126687/wheel/, 198 MB) was built on Thor for sm_110. dgx:gpu0 is sm_121. The one GB10 build attempt (job efc30c74, dequeued 2026-09-04T05:01:54Z) failed at three seconds of build wall:

ModuleNotFoundError: No module named 'setuptools_rust'

--no-build-isolation skipping the PEP-518 requires. A requeue with the requires installed is already staged by another session (requeue-prebuild.sh); it should be allowed to run rather than duplicated.

Two decisions this needs, and neither is an implementer's

  1. Does vllm_distribution_version get corrected to the source-build value? sync(UPSTREAM-SYNC-HEADPIN): step 6 owed POST-HOC at the new pin e126687a9a -- five rows, C1c, and a red means revert #2818 point 2 invites exactly this. Correcting it makes the runnable arm the gateable one.
  2. Should the check be equality at all? The comment above it records that a previous metadata == runtime == CONST shape was unsatisfiable at any value on this pin, and that startswith was rejected because it would accept an arbitrary suffix. There may be a third shape — matching the commit and treating the build-mode suffix as metadata rather than identity.

Filed to name the bind, not to prescribe. Do not "fix" it by loosening the check to make a red gate green (AGENTS.md §"Changing the rules or a checker"); the honest repair is to make the recorded string describe an install that can actually serve.

Not claimed

I have not established whether a source build at this pin produces a correct oracle on GB10 — only that it would be refused before it got the chance. Nor have I verified the 13,872-byte figure myself; it is read from the sync record.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions