Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix max length in run_plm script #8738

Merged
merged 1 commit into from
Nov 23, 2020
Merged

Fix max length in run_plm script #8738

merged 1 commit into from
Nov 23, 2020

Conversation

sgugger
Copy link
Collaborator

@sgugger sgugger commented Nov 23, 2020

What does this PR do?

The XLNet tokenizer has a ridiculously high maximum sequence length, so the run_plm was failing without setting the max_seq_length argument. This PR fixes that by setting default of 512 to it.

Fixes #8674

@sgugger sgugger mentioned this pull request Nov 23, 2020
Closed
3 tasks
LysandreJik
LysandreJik approved these changes Nov 23, 2020
View reviewed changes
Copy link
Member

@LysandreJik LysandreJik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@LysandreJik LysandreJik merged commit 367f497 into master Nov 23, 2020
@LysandreJik LysandreJik deleted the fix_8674 branch November 23, 2020 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LysandreJik LysandreJik LysandreJik approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Issues Fine-tuning XLNET
2 participants
@sgugger @LysandreJik