Coupling without Communication and Drafter-Invariant Speculative Decoding
-
Updated
Aug 17, 2024 - Python
Coupling without Communication and Drafter-Invariant Speculative Decoding
Some experiments aimed at increasing LLM throughput and efficiency via Speculative Decoding.
Dynasurge: Dynamic Tree Speculation for Prompt-Specific Decoding
Implementation of the paper Fast Inference from Transformers via Speculative Decoding, Leviathan et al. 2023.
Codes for our paper "Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation" (EMNLP 2023 Findings)
Verification of the effect of speculative decoding in Japanese.
Reproducibility Project for [NeurIPS'23] Speculative Decoding with Big Little Decoder
Implementation of Speculative Sampling in "Accelerating Large Language Model Decoding with Speculative Sampling"
"LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", Accepted to ACL 2024
SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration
[NeurIPS'23] Speculative Decoding with Big Little Decoder
[COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
scalable and robust tree-based speculative decoding algorithm
Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)
Large-scale LLM inference engine
⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡
Add a description, image, and links to the speculative-decoding topic page so that developers can more easily learn about it.
To associate your repository with the speculative-decoding topic, visit your repo's landing page and select "manage topics."