[Feature] [RFC] Add EDSD Support to SpecForge

[KerwinKai]

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/SpecForge/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Motivation

Motivation

We propose adding training support for EDSD (Entropy-Driven Speculative Decoding) to SpecForge. EDSD is an entropy-driven framework for compute- and data-efficient drafter training and design in speculative decoding, reducing training cost while improving acceptance and robustness.

Specifically, EDSD extends EAGLE3-style draft-model training with two components:

• EDTrain: an entropy-driven curriculum that gradually introduces harder tokens and increases the autoregressive rollout depth during training.
• EDArch: a draft architecture that selects two target-model hidden layers and fuses their features with token embeddings through EDFuse.

Paper:

https://aclanthology.org/2026.acl-long.2145/

Co-author: @Ximingwang-09

---

Proposed Changes

1. EDTrain

Add two training schedules:

• Entropy-based token masking: high-entropy positions are partially excluded in early epochs and gradually introduced later.
• Progressive rollout depth: the TTT simulation depth increases from a small value to the configured maximum during training.

Example arguments:

--drop-ratio-scale
--step-n-schedule 1 2 3 4 5 6 7

2. EDArch

Add an EdsdDraftModel with:

• two configurable target hidden-state layers;
• projection of the selected target features;
• EDFuse for combining target features and token embeddings;
• reuse of the existing EAGLE3 backbone, vocabulary mapping, and training infrastructure where possible.

Example configuration:

"target_layer_ids": [23, 34]

For the initial PR, the layer IDs would be provided in the model configuration. Automatic Curvature/LIDAR-based layer selection would remain outside the core training path.

3. SpecForge Integration

The prototype currently adds:

specforge/modeling/draft/edsd.py
specforge/core/edsd.py
scripts/train_edsd.py
configs/qwen3-8b-edsd.json
examples/run_qwen3_8b_edsd_online.sh

It reuses the existing SpecForge support for:

• online and offline training;
• Hugging Face and SGLang target backends;
• FSDP and tensor parallelism;
• checkpoint save and resume;
• existing EAGLE3 loss and data-processing utilities.

---

Initial PR Scope

Included:

• EDTrain scheduling;
• EDFuse and the EDSD draft model;
• configurable target-layer extraction;
• a Qwen3-8B example;
• unit tests and training smoke tests;
• documentation.

---

Validation

The PR will include:

• unit tests for the masking and rollout schedules;
• a short Qwen3-8B training smoke test;
• a controlled comparison with EAGLE3 under the same training setup;

---

This work was developed during an earlier phase of the recent rapid evolution of speculative decoding.

While the field has continued to advance, we hope that the entropy-driven curriculum, feature-fusion design, and layer-selection strategy introduced in EDSD may still offer useful insights and reusable components to the SpecForge community. We would be grateful for feedback from the maintainers and would be happy to adapt the implementation to better align with the project’s design and integration requirements.

Related resources

No response

[jiapingW]

This is a very interesting task. We are particularly interested in comparing the current methods EDSD and related methods in the DFlash family (DFlash, Domino, Dpsark). Could you demonstrate a comparison of your results with their mean accept length and other performance metrics on open-source models such as Qwen3-8B or other models?

[KerwinKai]

This is a very interesting task. We are particularly interested in comparing the current methods EDSD and related methods in the DFlash family (DFlash, Domino, Dpsark). Could you demonstrate a comparison of your results with their mean accept length and other performance metrics on open-source models such as Qwen3-8B or other models?

Thank you for your helpful suggestion. We are currently resolving the code conflicts and will then run additional experiments on open-source models such as Qwen3-8B. We plan to report the mean accepted length and other relevant performance metrics, and compare our results with methods from the DFlash family. We will update this issue once the results are available.