v0.1.2

EAGLE-4 0.1.2 updates eagle-harness on the 0.1.1 engines. On SGLang, EAGLE-4 decodes a batch without speculation wherever speculation would be slower, so the high-load recipe keeps pace with plain decoding up to 512 concurrent requests on four B300 GPUs and keeps its speedup at low load. eagle-harness calibrate --engine vllm completes on hybrid targets, and the TokenSpeed draft runs its candidate head on the engine's low-latency GEMM.

Install

You don't need to download anything from this page. pip fetches everything: eagle-harness comes from PyPI and the engines come from our wheel index. Install one engine per environment, with the plugin:

python3.12 -m venv .venv && source .venv/bin/activate
pip install "eagle-harness[sglang]" "sglang==0.5.23.dev252+eagle.5" \
  --extra-index-url https://eagle-hq.github.io/EAGLE-4/whl/cu130/ \
  --extra-index-url https://download.pytorch.org/whl/cu130 \
  --extra-index-url https://pypi.nvidia.com

To upgrade a 0.1.1 environment, run the same command with "eagle-harness[sglang]==0.1.2". The installation guide has the vLLM, TokenSpeed and ROCm steps. For high load, serve with the high-load recipe.

Measured throughput

4× B300, TP4, temperature 0. The speedup divides EAGLE-4 throughput by the same engine with speculation off, measured in the same allocation.

Engine and command	Concurrency	No-spec tok/s	EAGLE-4 tok/s	Speedup	Starts
SGLang, measured recipe	1	219.0	733.8	3.35×	2
SGLang, measured recipe	8	1545.2	3257.9	2.11×	2
SGLang, measured recipe	64	4718.6	6488.2	1.38×	2
SGLang, high-load recipe	64	7313.7	7393.3	1.01×	3
SGLang, high-load recipe	128	11086.4	11114.3	1.00×	3
SGLang, high-load recipe	256	14264.9	14332.3	1.00×	3
SGLang, high-load recipe	512	16340.7	16197.5	0.99×	3
vLLM, measured recipe	1	239.4	678.5	2.83×	4
vLLM, measured recipe	8	1668.9	2887.4	1.73×	4
vLLM, measured recipe	64	6725.4	8013.4	1.19×	4
TokenSpeed, measured recipe	1	283.5	886.5	3.13×	2
TokenSpeed, measured recipe	8	1628.2	3256.4	2.00×	2
TokenSpeed, measured recipe	64	4969.8	6974.1	1.40×	2

The high-load rows are restarts with warm caches; a first start on a machine measured 0.95× at 64 and 0.97× at 128. Every start also passes the twenty-prompt greedy correctness check. The release notes have the conditions.

Limits in this release

• The SGLang quick-start command is slower than plain decoding above about 32 to 64 concurrent requests; for high load, use the high-load recipe on four GPUs. A fix is planned.
• High load on one GPU is not supported yet.
• vLLM and TokenSpeed do not switch to plain decoding at high load yet; above concurrency 64, serve them without speculation for now.
• Known limits lists the rest.

Files

No files on this page. The engine wheels are on the v0.1.0 release, which our wheel index serves, and eagle-harness is on PyPI. The :0.1.2 images (ghcr.io/eagle-hq/eagle-4-sglang:0.1.2, ghcr.io/eagle-hq/eagle-4-vllm:0.1.2, ghcr.io/eagle-hq/eagle-4-tokenspeed:0.1.2) carry eagle-harness 0.1.2.

v0.1.1

EAGLE-4 0.1.1 updates eagle-harness and two engine builds. An SGLang server now needs only the draft path to serve EAGLE-4 and starts its first requests faster, and a new SGLang CUDA engine adds rounds without speculation where the live costs pick them and a temperature-1 sampled tree. vLLM measures the tree acceptance it serves, and the TokenSpeed integration moves to a new engine build. The vLLM engine is unchanged.

Install

You don't need to download anything from this page. pip fetches everything: eagle-harness comes from PyPI and the engines come from our wheel index. Install one engine per environment, with the plugin:

python3.12 -m venv .venv && source .venv/bin/activate
pip install "eagle-harness[sglang]" "sglang==0.5.23.dev252+eagle.5" \
  --extra-index-url https://eagle-hq.github.io/EAGLE-4/whl/cu130/ \
  --extra-index-url https://download.pytorch.org/whl/cu130 \
  --extra-index-url https://pypi.nvidia.com

To upgrade a 0.1.0 environment, run the same command with "eagle-harness[sglang]==0.1.1". The installation guide has the vLLM, TokenSpeed and ROCm steps.

Breaking: a config that still sets "mode": "eval", ragged, cost_model.ragged_* or draft_model.moe_gate_acc stops at load. Delete the option; the migration guide lists each one.

Measured throughput

4× B300, TP4, temperature 0. The speedup divides EAGLE-4 throughput by the same engine with speculation off, measured in the same allocation.

Engine	Concurrency	No-spec tok/s	EAGLE-4 tok/s	Speedup	Starts
SGLang	1	218.5	750.6	3.44×	3
SGLang	8	1541.7	3203.1	2.08×	3
SGLang	64	4655.3	6454.1	1.39×	3
SGLang, 32K context	1	239.7	715.3	2.98×	4
vLLM	1	240.5	681.0	2.83×	6
vLLM	8	1686.7	2894.8	1.72×	6
vLLM	64	6950.5	7935.3	1.14×	6

Every start also passes the twenty-prompt greedy correctness check. The release notes have temperature 1 and the conditions.

Limits in this release

• SGLang above concurrency 64 is slower with speculation than without at a running cap of 512; serve those loads without speculation for now.
• TokenSpeed runs on NVIDIA CUDA only.
• Known limits lists the rest.

Files

The images are ghcr.io/eagle-hq/eagle-4-sglang:0.1.1, ghcr.io/eagle-hq/eagle-4-vllm:0.1.1 and ghcr.io/eagle-hq/eagle-4-tokenspeed:0.1.1. No other files on this page. The new engine wheels are on the v0.1.0 release, which our wheel index serves, and eagle-harness is on PyPI.

v0.1.0

EAGLE-4 0.1.0 is the first public release of the EAGLE-4 inference code. One drafter sizes its speculative tree from measured draft and verification costs, so it adapts to the batch size, the hardware and the workload, and it keeps the target model's output distribution.

eagle-harness installs as a plugin into patched SGLang, vLLM and TokenSpeed builds. It registers through hooks the engine exposes and does not patch the engine at runtime. The measured target pair is Qwen3.8-27B with its EAGLE-4 draft.

Install

You don't need to download anything from this page. pip fetches everything: eagle-harness comes from PyPI and the patched engine comes from our wheel index. Install one engine per environment, then the plugin:

git clone https://github.com/eagle-hq/EAGLE-4 && cd EAGLE-4
. ./docs/release.env
python3.12 -m venv .venv && source .venv/bin/activate
python -m pip install --extra-index-url "$EAGLE_CUDA_INDEX" \
  --extra-index-url "$EAGLE_TORCH_CUDA_INDEX" "sglang==$EAGLE_SGLANG_VERSION" 'flashinfer-python>=0.7.2'
python -m pip install "eagle-harness==$EAGLE_VERSION"
eagle-harness-check

The installation guide has the vLLM and TokenSpeed steps. The engine sources are public at eagle-hq/sglang, eagle-hq/vllm and eagle-hq/tokenspeed.

Measured throughput

Generated tokens per second, mean of two server starts, SPEED-Bench client, temperature 0. The speedup divides EAGLE-4 throughput by the same engine with speculation off.

Engine	GPUs	Concurrency	No-spec tok/s	EAGLE-4 tok/s	Speedup
SGLang	4× B300, TP4	8	1479.4	2820.6	1.91×
SGLang	4× B300, TP4	64	3925.3	5281.3	1.35×
SGLang	1× MI355X, TP1	8	552.7	1145.9	2.07×
SGLang	1× MI355X, TP1	32	1269.2	1980.8	1.56×
vLLM	4× B300, TP4	1	241.6	722.8	2.99×
vLLM	4× B300, TP4	8	1699.9	2883.9	1.70×

Every row also passes the greedy correctness check. The support matrix has the full results and their conditions.

Limits in this release

• TokenSpeed is experimental, NVIDIA CUDA only, and has no throughput claim.
• The AMD ROCm wheels are still being verified.
• Known limits lists the configurations still below target.

Files on this page

The five engine wheels (sglang, vllm, tokenspeed*) are the files our wheel index serves, and engine-cu130-SHA256SUMS lists their checksums so you can verify a download with sha256sum -c. eagle-harness itself is on PyPI.