This repository has been archived by the owner on Nov 22, 2022. It is now read-only.
Consolidate BERT, XLM and RobERTa Tensorizers #1119
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D18290264 |
kartikayk
force-pushed
the
export-D18290264
branch
from
06f528c
to
2c9d5a1
Compare
Summary: Pull Request resolved: facebookresearch#1119 In this diff I take a fast stab at consolidating the XLM, BERT and RoBERTa Tensorizers. I kill a bunch of dead code and simiplify a lot. - I create a BERTTensorizerBase class which derives from Tensorizer and not TokenTensorizer since this makes the logic a lot easier especially since we no longer have to deal with all the bos, eos flags. Given that tokenize and lookup_tokens are not part of TokenTensorizer, I think this formulation makes a lot of sense. - As per suggestions, I derive the config classes from Tensorizer as well and kill all of the special flags. - I try to put as much of the functionality in the base class as possible in order to minimize copy paste code. There is still some but I dont want perfect to be the enemy of better. - I kill TLM - long live TLM. - I (temporaarily) kill support for OSS XLM which probably should have its own tensorizer anyways since it has nothing to do with transformer_sentence_encoder. Reviewed By: rutyrinott Differential Revision: D18290264 fbshipit-source-id: 573970838808399b6534dad1464057ebd9cb6d44
kartikayk
force-pushed
the
export-D18290264
branch
from
2c9d5a1
to
6236f4c
Compare
|
This pull request was exported from Phabricator. Differential Revision: D18290264 |
|
This pull request has been merged in 5b83da0. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
In this diff I take a fast stab at consolidating the XLM, BERT and RoBERTa Tensorizers. I kill a bunch of dead code and simiplify a lot.
Reviewed By: rutyrinott
Differential Revision: D18290264