dflash-llama

Adapt DFlash for llama.cpp, so DFlash speculative decoding can run against GGUF target models.

This repository is currently a planning and scaffolding repo. It intentionally does not claim to contain a working DFlash runtime yet.

Goal

DFlash is a block-diffusion draft-model speculative decoding approach. The goal of dflash-llama is to make that approach usable from the llama.cpp ecosystem:

• target model loaded as GGUF through llama.cpp
• DFlash draft model loaded as GGUF
• hidden-state capture from the target model
• DFlash draft generation over a fixed block
• target-model verification with exact-output speculative decoding
• eventual integration with llama.cpp speculative decoding APIs

Core idea

The target GGUF remains the source of truth for tokens, embeddings, logits, and verification. The DFlash draft GGUF predicts a block of candidate tokens from:

1. the last accepted token plus mask-token noise embeddings
2. selected hidden states captured from the target model
3. Qwen3-style DFlash transformer blocks

The target model verifies the candidate block and accepts the longest matching prefix, preserving exact greedy output.

Planned architecture

The recommended implementation path is:

1. Standalone dflash-llama repository with llama.cpp as a pinned submodule.
2. Initial hidden-state capture through llama.cpp evaluation callbacks where possible.
3. DFlash GGUF converter for upstream DFlash checkpoints.
4. C++ DFlash graph/runtime built with ggml and llama.cpp backend support.
5. End-to-end llama-dflash-cli for prompt, draft, verify, accept, rollback.
6. Later upstream-friendly patches for generic auxiliary layer capture and --spec-type dflash.

See docs/architecture.md for the full implementation plan.

Status

• Project plan
• Repository scaffold
• DFlash GGUF metadata specification
• HF-to-GGUF converter
• C++ DFlash loader
• C++ graph builder
• Target hidden-state capture
• End-to-end speculative loop
• Benchmarks
• llama.cpp upstream patches

Repository layout

dflash-llama/
├── docs/
│   ├── architecture.md
│   ├── gguf_layout.md
│   ├── roadmap.md
│   └── upstream_status.md
├── include/
│   └── llama-dflash.h
├── src/dflash/
├── tools/
├── python/dflash_llama/
├── tests/
├── bench/
├── scripts/
├── patches/
└── third_party/llama.cpp/   # planned submodule

License

MIT for this repository's original code and documentation. Upstream projects and model weights retain their own licenses.