This repository has been archived by the owner on Nov 22, 2022. It is now read-only.
Integrate XLM-R into PyText #1120
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D18382033 |
kartikayk
added a commit
to kartikayk/pytext
that referenced
this pull request
Nov 8, 2019
Summary: Pull Request resolved: facebookresearch#1120 Adding the ability to load and finetune XLM-R models in PyText. Reviewed By: rutyrinott Differential Revision: D18382033 fbshipit-source-id: bd111ad80d8c6369b6dc8957bbfab0f1c1c9eabb
kartikayk
force-pushed
the
export-D18382033
branch
from
e4df5c1
to
2198c93
Compare
|
This pull request was exported from Phabricator. Differential Revision: D18382033 |
Summary: In this diff I take a fast stab at consolidating the XLM, BERT and RoBERTa Tensorizers. I kill a bunch of dead code and simiplify a lot. - I create a BERTTensorizerBase class which derives from Tensorizer and not TokenTensorizer since this makes the logic a lot easier especially since we no longer have to deal with all the bos, eos flags. Given that tokenize and lookup_tokens are not part of TokenTensorizer, I think this formulation makes a lot of sense. - As per suggestions, I derive the config classes from Tensorizer as well and kill all of the special flags. - I try to put as much of the functionality in the base class as possible in order to minimize copy paste code. There is still some but I dont want perfect to be the enemy of better. - I kill TLM - long live TLM. - I (temporaarily) kill support for OSS XLM which probably should have its own tensorizer anyways since it has nothing to do with transformer_sentence_encoder. Differential Revision: D18290264 fbshipit-source-id: 7746e87fa350a31cea82b063be9151b7e8a04f71
Summary: Pull Request resolved: facebookresearch#1120 Adding the ability to load and finetune XLM-R models in PyText. Reviewed By: rutyrinott Differential Revision: D18382033 fbshipit-source-id: 157a53fb44b46452fed7005db9682c9dc46f28da
kartikayk
force-pushed
the
export-D18382033
branch
from
2198c93
to
ce61992
Compare
|
This pull request was exported from Phabricator. Differential Revision: D18382033 |
|
This pull request has been merged in 8a88897. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: Adding the ability to load and finetune XLM-R models in PyText.
Reviewed By: rutyrinott
Differential Revision: D18382033