Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add custom stop token ids for generation (by making eos_token_id List[int]) #1

Merged
merged 5 commits into from
Dec 31, 2022
Merged

Add custom stop token ids for generation (by making eos_token_id List[int]) #1

merged 5 commits into from
Dec 31, 2022

Commits on Dec 19, 2022

  1. Add support for list of ints for eos_token_id inside generation/utils.py

    @tokestermw
    tokestermw committed Dec 19, 2022
    Configuration menu
    Copy the full SHA
    70c2dda View commit details
    Browse the repository at this point in the history

  2. Instead of having if elses, cast the eos_token_id into a List[int]

    @tokestermw
    tokestermw committed Dec 19, 2022
    Configuration menu
    Copy the full SHA
    b0cccd6 View commit details
    Browse the repository at this point in the history

  3. Add List[int] support for logits_process.py

    @tokestermw
    tokestermw committed Dec 19, 2022
    Configuration menu
    Copy the full SHA
    fba3345 View commit details
    Browse the repository at this point in the history

  4. add List[int] for beam_search.py

    @tokestermw
    tokestermw committed Dec 19, 2022
    Configuration menu
    Copy the full SHA
    405f79c View commit details
    Browse the repository at this point in the history

  5. add List[int] for forced_eos_token_id

    @tokestermw
    tokestermw committed Dec 19, 2022
    Configuration menu
    Copy the full SHA
    8298e87 View commit details
    Browse the repository at this point in the history