Publish the GB10 runtime base and make DeepSeek runnable from it - #63
Merged
Conversation
The runtime layer the serving profiles derive from is retrievable from a registry: `ghcr.io/fujitsupolycom/gb10-vllm-base` at manifest digest `sha256:9d88c2152b0ae9f33e7a793b7df29398ed79710b205b9244ac63597ab4481ada`, `linux/arm64`, built from source commit `19523482c298`. `runtime/faststart-lock.json` records it, so it is named by content digest like every other pinned input, and `scripts/pull_pinned_images.py` retrieves it alongside them. Two documents asked a reader for an image and did not say where to get one. `runtime/exl3-r7/README.md` required a parent supplied through `BASE_IMAGE`, and `docs/DEEPSEEK_V4_FLASH_QUICKSTART.md` required "a vLLM build that can load `DeepseekV4ForCausalLM` with the B12X kernel family". Both now give the pull. Scope. The published image is the runtime base, not a serving profile: a deployment still builds its derived image. The digest identifies one immutable set of bytes; keeping them served is a separate ongoing act, which `docs/RUNTIME_INPUT_DURABILITY.md` states. Validation: ruff, `python -m pytest scripts runtime -q` at 1789 passed 12 skipped, `python scripts/pull_pinned_images.py --plan` listing four digest-pinned images, and the repo-relative Markdown link and anchor check.
The quickstart asked for "a vLLM build that can load
`DeepseekV4ForCausalLM` with the B12X kernel family" without saying where
to obtain one, and its launch command omitted four flags the running
configuration carries.
The published GB10 runtime base registers `DeepseekV4ForCausalLM` and
carries B12X, the patched NCCL, and the transport library, so it serves
this checkpoint with no derived build. The launch is a container
invocation against that image by digest rather than a bare `vllm serve`,
including the host namespaces, RDMA device, and memory-lock limit the
transport needs.
Four flags are stated as load-bearing because omitting any of them fails
a deployment that is otherwise correct:
- `--kernel-config '{"enable_cutedsl_warmup": false}'` disables a CuteDSL
router-GEMM warmup that aborts the worker before the engine starts.
- `--kv-cache-dtype fp8_ds_mla` declares the layout the engine allocates.
`fp8` allocates identically and serves identically, and declares a
geometry that does not match, which an external key-value consumer
reads.
- `--tokenizer-mode deepseek_v4`.
- `--headless` on ranks 1 through 3, without which every rank binds the
API port.
Validation: ruff, `python -m pytest scripts runtime -q` at 1789 passed
12 skipped, and the repo-relative Markdown link and anchor check. The
architecture registration was checked by running the published image:
`DeepseekV4ForCausalLM` is among the 360 architectures it registers.
…s it Two images are published and pinned by digest in `runtime/faststart-lock.json`: - `ghcr.io/fujitsupolycom/gb10-vllm-base` — the runtime layer the serving profiles derive from. SM121 kernels, aarch64 cu132 wheels, the pinned vLLM, B12X, FlashInfer, DeepGEMM, the patched NCCL, and `libspark_transport_capi.so` with fourteen probe binaries. It is the parent `runtime/exl3-r7/build-image.sh` takes through `BASE_IMAGE`. - `ghcr.io/fujitsupolycom/gb10-vllm-serving` — derived from that base and carrying LMCache, the EXL3 quantization overlay, and the R7 profile overlays. It registers `Glm4MoeForCausalLM` and `DeepseekV4ForCausalLM`. A four-rank launch established which image serves DeepSeek-V4-Flash-0731 and corrected the quickstart accordingly. The base registers the architecture but does not serve the checkpoint: its vLLM build rejects `--kernel-config`, and its entrypoint requires GLM attestation variables that this checkpoint has no values for. The quickstart names the serving image. Both images run an attestation entrypoint, so a launch for a checkpoint other than the profile an image attests overrides it with `--entrypoint`. The quickstart's command does. `scripts/pull_pinned_images.py` retrieves both alongside the other pinned images, five in total. Validation: ruff, `python -m pytest scripts runtime -q` at 1789 passed 12 skipped, `pull_pinned_images.py --plan`, and the repo-relative Markdown link and anchor check.
FujitsuPolycom
force-pushed
the
claude/publish-runtime-base
branch
from
August 20, 2026 06:11
39a0330 to
b125efe
Compare
`ghcr.io/fujitsupolycom/gb10-vllm-serving` at digest `sha256:df0e2068fc7034a1ec7a2c1fa4e0c3224c720161539525b5a7cbb037dc1d0f8e`, `linux/arm64`, is the published runtime. It carries the pinned vLLM, B12X, LMCache, the EXL3 quantization overlay, FlashInfer, DeepGEMM, the patched NCCL, and `libspark_transport_capi.so` with its probe binaries, and registers both `Glm4MoeForCausalLM` and `DeepseekV4ForCausalLM`. One image covers the GLM profiles and DeepSeek-V4-Flash-0731. `runtime/faststart-lock.json` records it by digest and `scripts/pull_pinned_images.py` retrieves it. A four-rank launch established which image serves this checkpoint. The runtime base layer registers the architecture but does not serve it: its vLLM build rejects `--kernel-config`, and its entrypoint requires GLM attestation variables that the checkpoint has no values for. Publishing that layer separately would have offered a reader a second image that looks equivalent and is not, so only the derived image is published. The bootstrap still builds the base layer locally, and `runtime/exl3-r7/build-image.sh` still takes a parent through `BASE_IMAGE`. The published image runs an entrypoint that attests a GLM profile, so a launch for another checkpoint overrides it with `--entrypoint`. The DeepSeek quickstart's command does. Validation: ruff, `python -m pytest scripts runtime -q` at 1789 passed 12 skipped, `pull_pinned_images.py --plan`, and the repo-relative Markdown link and anchor check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resulting behavior
The runtime layer the serving profiles derive from is published and pinned by
digest, and the DeepSeek quickstart launches from it.
ghcr.io/fujitsupolycom/gb10-vllm-base@sha256:9d88c2152b0ae9f33e7a793b7df29398ed79710b205b9244ac63597ab4481ada—
linux/arm64, 23.6 GB, built from source commit19523482c298. Anonymouspull verified.
What the image carries
Checked by running it, not inferred:
0.11.2.dev279+eldritch.final.fcc6141.b12x284a2ea.fi25dd814.cu132.20260626,registering 360 architectures including
DeepseekV4ForCausalLMandGlm4MoeForCausalLM0.6.13+cu132, DeepGEMM2.5.0, torch2.12.0+cu132libspark_transport_capi.soand fourteen probe binaries — the ARM64 CUDAbuild a deployment would otherwise perform
libnccl.so.2.30.7/opt/spark-vllm/sitecustomize.py, so the transport installs when aVLLM_SPARK_TP4_*mode is setNot carried, and needed only by the EXL3 lanes: ExLlamaV3 and LMCache, which
scripts/bootstrap_exl3.pybuilds into a derived image.Two dead ends replaced
runtime/exl3-r7/README.mdrequired a parent image supplied throughBASE_IMAGEwith no way to obtain one.docs/DEEPSEEK_V4_FLASH_QUICKSTART.mdrequired "a vLLM build that can load
DeepseekV4ForCausalLMwith the B12Xkernel family" and did not say where to get it. Both now give the pull.
The DeepSeek launch matches the running configuration
The quickstart's command omitted four flags the deployed stack carries. Each
failure is one an operator would hit after provisioning four machines:
--kernel-config '{"enable_cutedsl_warmup": false}'— without it the workeraborts in CuteDSL router-GEMM warmup before the engine starts
--kv-cache-dtype fp8_ds_mla—fp8allocates and serves identically whiledeclaring a geometry that does not match what is allocated; the engine's own
kernels never read that declaration, an external key-value consumer does
--tokenizer-mode deepseek_v4--headlesson ranks 1 through 3The launch is now a container invocation carrying the host namespaces, RDMA
device, and memory-lock limit the transport requires.
Lock and retrieval
runtime/faststart-lock.jsonrecords the image by manifest digest, andscripts/pull_pinned_images.pyretrieves it with the other pinned images —four in total.
check_runtime_inputs.py --check-remotewatches it forretrievability alongside the git sources.
Scope
The published image is the runtime base, not a serving profile. A digest states
which bytes are correct; keeping them served is a separate ongoing act, which
docs/RUNTIME_INPUT_DURABILITY.mdstates.Validation
ruffclean.python -m pytest scripts runtime -q→ 1789 passed, 12 skipped.pull_pinned_images.py --planlists four digest-pinned images. Repo-relativeMarkdown link and anchor check passes.