Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Inference/SpecDec] Add Speculative Decoding Implementation #5423

Merged
merged 9 commits into from
Mar 11, 2024
Merged

[Inference/SpecDec] Add Speculative Decoding Implementation #5423

merged 9 commits into from
Mar 11, 2024

Commits on Mar 2, 2024

  1. fix flash decoding mask during verification

    @yuanheng-zhao
    yuanheng-zhao committed Mar 2, 2024
    Configuration menu
    Copy the full SHA
    af5c146 View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2024

  1. add spec-dec

    @yuanheng-zhao
    yuanheng-zhao committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    a204c95 View commit details
    Browse the repository at this point in the history

  2. add test for spec-dec

    @yuanheng-zhao
    yuanheng-zhao committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    5625256 View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2024

  1. revise drafter init

    @yuanheng-zhao
    yuanheng-zhao committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    a7839f3 View commit details
    Browse the repository at this point in the history

  2. remove drafter sampling

    @yuanheng-zhao
    yuanheng-zhao committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    fb7ac75 View commit details
    Browse the repository at this point in the history

  3. retire past kv in drafter

    @yuanheng-zhao
    yuanheng-zhao committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    4f29eb0 View commit details
    Browse the repository at this point in the history

  4. (trivial) rename attrs

    @yuanheng-zhao
    yuanheng-zhao committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    768cc8d View commit details
    Browse the repository at this point in the history

  5. (trivial) rename arg

    @yuanheng-zhao
    yuanheng-zhao committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    7bebed0 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2024

  1. revise how we enable/disable spec-dec

    @yuanheng-zhao
    yuanheng-zhao committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    88b877e View commit details
    Browse the repository at this point in the history