Skip to content

Honor the DFlash draft load config - #134

Closed
FujitsuPolycom wants to merge 1 commit into
codex/cuda-restore-terminologyfrom
codex/dflash-draft-load-config
Closed

Honor the DFlash draft load config#134
FujitsuPolycom wants to merge 1 commit into
codex/cuda-restore-terminologyfrom
codex/dflash-draft-load-config

Conversation

@FujitsuPolycom

Copy link
Copy Markdown
Owner

Resulting behavior

The composed vLLM 0b67266a0f37d6146a8403fb8482403c62f412d5 runtime passes SpeculativeConfig.draft_load_config into the DFlash get_model() call. A profile can therefore keep global target loading on fastsafetensors while loading the external BF16 DFlash2 checkpoint with standard safetensors.

The exact configuration shape is:

--load-format fastsafetensors
--speculative-config '{"method":"dflash","model":"/mtp-draft","num_speculative_tokens":7,"draft_tensor_parallel_size":4,"kv_cache_dtype":"auto","draft_sample_method":"probabilistic","rejection_sample_method":"standard","draft_load_config":{"load_format":"safetensors"}}'

draft_load_config remains optional. When it is absent or null, vLLM's existing get_model(load_config or vllm_config.load_config) rule retains the global target loader for the draft.

Exact source contract

  • vLLM commit: 0b67266a0f37d6146a8403fb8482403c62f412d5
  • vLLM tree: ba9484ccb33aa56e90ff2f447f15ca9b9da97639
  • target: vllm/v1/worker/gpu/spec_decode/dflash/utils.py
  • preimage SHA-256: 2301c8199b73ed893dfbd3ae14ad125816f100b2d2ed034215b1f2d9aa2c23c5
  • patch SHA-256: 39b567013ee7aed79f63200ed460129587933dc77fb430decdf19f78178de279
  • postimage SHA-256: 98acbae2b3bb4482d83f9637c163ce7c92707ccdf6561b7e431f23337f151cf4

The source preparer applies and reverses the patch against the immutable vLLM checkout to prove both byte boundaries. The prepared-context receipt, image labels, composed verifier, and profile label requirements bind the patch and postimage. GPU-free AST checks reject a DFlash loader that ignores draft_load_config and reject a model-loader implementation that loses the None fallback.

Compatibility

This draft was created from exact SparkRing #131 commit a7e5a70af1b7cb329651712a449c60f54b1d0ae5. The #131 branch later added unrelated README commit 56b3a8f77d5b35a63bb97c213a73d34e5c178db6; that commit is deliberately not folded into this exact-base branch.

No vLLM, B12X, SparkCache, NCCL, model, or image source commit changed. Existing MTP profile identity fields and SparkCache cache identity are unchanged. Cache namespace impact: none.

No image was built or published. No service or serving host was contacted or changed.

Validation

  • Focused runtime, resolver, terminology, and launcher tests: 42 passed.
  • DFlash runtime and resolver tests: 23 passed.
  • Fresh source-context preparation followed by independent receipt verification: passed.
  • Ruff, JSON parsing, Python compilation, Bash syntax, and diff checks: passed.
  • Maintained exact-base suite: 1,935 passed, 9 skipped, 1 inherited README assertion failure. The later Use SparkCache CUDA restore terminology #131 commit named above addresses that unrelated assertion outside this requested base.

Patch the exact vLLM 0b67266 DFlash loader so get_model receives SpeculativeConfig.draft_load_config. An explicit draft LoadConfig can select safetensors while the target retains global fastsafetensors; None preserves the target LoadConfig fallback.

Bind the preimage, patch, and postimage in the source-context receipt, image labels, composed verifier, profile requirements, and GPU-free semantic tests. Source commits, cache identity, and SparkCache namespace are unchanged. Validation: 42 focused tests passed; end-to-end context preparation and re-verification, Ruff, JSON, Python, Bash, and diff checks passed. The maintained suite retains one unrelated base README failure after 1,935 passes and 9 skips.
@FujitsuPolycom

Copy link
Copy Markdown
Owner Author

The resulting runtime, operator contract, and evidence are consolidated in retained draft stack #146#147#150. Independent page-base research remains in #149. Closing this superseded draft and deleting only its remote head branch.

@FujitsuPolycom
FujitsuPolycom deleted the codex/dflash-draft-load-config branch August 31, 2026 01:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant