Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move
eos_token_id
to stopping criteria #29459Move
eos_token_id
to stopping criteria #29459Changes from 5 commits
584de18
79a47c4
6c93f8b
f59b83f
8ebad2d
3e2507b
b77b6ab
edc76df
f71a687
387be0e
14ece04
bc3eea9
7518aea
8e5ec57
2544d12
12acbc4
1107673
1ffc554
ca0a414
ce093c1
aa7fae4
5375d97
d103d29
9f59abb
48f33fc
801af07
a385c6d
7c00bb1
530064d
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would be nice to also support list of lists ? To have stopping tokens (if the eos is
["<","eos",">"]
)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, is it really true for existing models to have eos tokens as a sequence? I guess you are referring to custom eos tokens, when users want to stop at "" but they haven't trained the model with "" as special token. If that's the case, there is
StopStringsCriteria
PR coming or users are free to write their custom criteria@gante wdyt?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, the case where we want to stop on a string will be covered by #28932 (and is way more complex)
This PR is exclusively to port the existing EOS logic into its own stopping criteria :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right I forgot it was being covered