-
Notifications
You must be signed in to change notification settings - Fork 118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Resolve #218: Support soft ids as inputs for BERT/GPT2/RoBERTa/XLNet #220
Conversation
Resolve #218 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. What worries me is that our implementation of modules based on pre-trained stuff is a bit too repetitive, so that a seemingly small change would require modifying a bunch of files. This is also true for a lot of tests (not limited to pre-trained ones). Let's keep this in mind so we can improve this in the future.
This is a very good suggestion. For the code redundancy in the models, I feel that we can extract some task-specific headers for all or most of the pre-trained models. For the testing issue, some of the tests can be shared across different models. I think we can write a common test script that can be used by a group of classes for unit testing. |
That sounds good. Let's open a separate issue to keep track of this, so we can merge this PR. |
Feel free to merge when you're done with the changes. |
Support
soft_ids
as inputs forencoders
,classifiers
, andregressors
.