-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rinna RoBERTa's max_length is 510 not 512? #3
Comments
Hi @masayakondo, the maximum length of So I believe there should not be any errors inputting a 512-length sequence of tokens. Could you please share the code that causes the error? Thanks! |
Hi @ZHAOTING , thank you for your reply.
In the case of RoBERTa, from the following code, I thought that padding_idx and position_embeddings, or padding_idx and sentence length, were related. I am very sorry if my comment is misguided. Thanks! |
You are correct about huggingface's roberta code! I didn't notice how they construct To properly use our model, please try constructing
|
Yeah, I think you're right, too...
Thanks you for the advice. I will refer to it. Thanks! |
Hi, I have been using rinna RoBERTa for a while now.
I have a question.
The max_length of rinna RoBERTa is 510 (not 512), right?
Is this the intended result? If this was the intended result, why did you use 510 instead of 512 for max_length?
rinna RoBERTa's padding_idx is 3 (not 1). So I think the starting position of position_embeddings is padding_idx+1=4 as in the following problem, but the size of position_embeddings in rinna RoBERTa is (514, 768). If I actually enter text with a length of 512, I get an index error.
The text was updated successfully, but these errors were encountered: