Publish the reproducible GLM-5.3 R8 ARM64 runtime - #153
Merged
Conversation
Plan one four-rank direct-link chain without remote work, verify SHA-256 on every rank, resume bounded partial transfers, and optionally import only the expected image ID. Conflicting final files remain unchanged and interrupted operations retain resumable evidence. This adds no runtime-profile, cache-identity, or serving behavior.
Select 26 GiB of FP8 KV for DCP1 and 30 GiB for DCP2/DCP4 when KV_CACHE_MEMORY_BYTES=auto. Add a SparkCache enable switch so one image can run persistent SparkCache or vLLM prefix caching alone. Record the 942,767-token DCP1 result and the 942,592-token publication with exact artifact identity and limitations. Persisted cache identity and formats are unchanged. Validation: 28 focused tests passed; Ruff, shell syntax, JSON parsing, and diff checks passed.
The benchmark receipt already verifies the excluded capacity-limited cells. Public main removed the duplicate README sentence but retained the old assertion, leaving the repository suite red. Validation: 1,956 tests passed, 9 skipped; Ruff and diff checks passed.
Document that disabling SparkCache retains vLLM prefix caching and has launcher regression coverage but no retained live run for the exact image. Validation: 28 focused tests passed; diff checks passed.
Pin the image builder and OCI source labels to SparkCache main c3887f34, which contains the reviewed CUDA restore, page-delta, shared-base, DCP identity, generic-profile, evidence, and documentation stack. The retained local image receipt still identifies the preceding artifact until the replacement build is verified. Validation: 28 focused tests passed; Ruff and diff checks passed.
Set the launcher and environment template to the immutable GHCR digest, document pull-once direct-fabric fan-out, and record construction, distribution, vLLM-only, DCP1/DCP2/DCP4 restart restore, and one-million-depth evidence for image ID b3a13d80. The deep 942592-token snapshot was published but not replayed. Validation: 30 focused tests passed; Ruff, JSON, shell syntax, and diff checks passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Result
SparkRing can build, verify, distribute, and launch one GLM-5.3 Flash R8
Linux/ARM64 image with SparkCache across four GB10 systems. The launcher uses
one environment file for TP4 with DCP1, DCP2, or DCP4 and exposes the 1M
context limit, 8K batched-token budget, topology-aware KV allocation, model
paths, cache limits, speculation, ports, and network settings. One switch
selects persistent SparkCache or vLLM's GPU prefix cache alone.
The immutable Linux/ARM64 image is published at
ghcr.io/fujitsupolycom/sparkring-glm53-sparkcache@sha256:380283a506aeb8f9d486a3c64cd738e44268c3cc21590913ea9e4685869f256a.Its local Docker image ID is
sha256:b3a13d8003e7de30d7737fd33c8307404e506ba570240819ec7eb4f5c611400f.Technical reason
The R8 scheduler, DCP-aware SparkCache geometry, CUDA placement library,
switchless NCCL, and B12X runtime must remain one source- and image-bound
composition. A single launcher prevents DCP1, DCP2, and DCP4 from silently
selecting incompatible KV interleaving or full-CKV gather settings.
The direct-fabric image utility downloads one checksum-bound archive, forwards
it through adjacent links, verifies every copy, imports it on all four ranks,
and checks the local image ID.
Compatibility
manager-pages-v2namespace and defaults to completesnapshot-v1publication.SPARKCACHE_ENABLED=0omits persistent publication and restore withoutdisabling vLLM prefix caching.
Local Inference Lab provides the GLM-5.3 NVFP4 target quantization, Jovian
Judgement vLLM work, and B12X GB10 kernel integration. The external BF16 draft
comes from
incoai/GLM-5.3-Flash-DFlash2. Exact revisions are inruntime/glm53-flash-jj-r8-gb10/pins.json.Validation
python -m pytest runtime/glm53-flash-jj-r8-gb10 scripts/test_fanout_image_archive.py -q— 30 passed.
python -m ruff check runtime/glm53-flash-jj-r8-gb10 scripts/fanout_image_archive.py scripts/test_fanout_image_archive.py— passed.
python -m pytest spark_transport runtime scripts performance/harnesses -q -rs— 1,958 passed, 9 skipped.
python -m ruff check spark_transport runtime scripts performance/harnesses— passed.
links and loaded with the same image ID on all four ranks.
replacement, verified SparkCache CUDA restore, and a semantic marker check.
SparkCache connector.
with 26 GiB of FP8 KV per rank and published a 942,592-token SparkCache
snapshot.
The large snapshot was not replayed after process replacement. The retained
requests do not establish concurrent large-context restore, long-duration
behavior, or general throughput.