Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add new pre-trained models BERTweet and PhoBERT #6129

Merged
merged 34 commits into from Sep 18, 2020
Merged

Add new pre-trained models BERTweet and PhoBERT #6129

merged 34 commits into from Sep 18, 2020

Commits on Jul 29, 2020

  1. Add BERTweet and PhoBERT models

    @datquocnguyen
    datquocnguyen committed Jul 29, 2020
    Configuration menu
    Copy the full SHA
    b2b24d1 View commit details
    Browse the repository at this point in the history

  2. Update modeling_auto.py 

    Re-add `bart` to LM_MAPPING
    @datquocnguyen
    datquocnguyen committed Jul 29, 2020
    Configuration menu
    Copy the full SHA
    b30bcdf View commit details
    Browse the repository at this point in the history

  3. Update tokenization_auto.py 

    Re-add `from .configuration_mobilebert import MobileBertConfig`
not sure why it's replaced by `from transformers.configuration_mobilebert import MobileBertConfig`
    @datquocnguyen
    datquocnguyen committed Jul 29, 2020
    Configuration menu
    Copy the full SHA
    aa2e5b5 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2020

  1. Add BERTweet and PhoBERT to pretrained_models.rst

    @datquocnguyen
    datquocnguyen committed Jul 30, 2020
    Configuration menu
    Copy the full SHA
    a49233f View commit details
    Browse the repository at this point in the history

  2. Update tokenization_auto.py 

    Remove BertweetTokenizer and PhobertTokenizer out of tokenization_auto.py (they are currently not supported by AutoTokenizer.
    @datquocnguyen
    datquocnguyen committed Jul 30, 2020
    Configuration menu
    Copy the full SHA
    c51345a View commit details
    Browse the repository at this point in the history

  3. Update BertweetTokenizer - without nltk

    @datquocnguyen
    datquocnguyen committed Jul 30, 2020
    Configuration menu
    Copy the full SHA
    2cf49f9 View commit details
    Browse the repository at this point in the history

  4. Update model card for BERTweet

    @datquocnguyen
    datquocnguyen committed Jul 30, 2020
    Configuration menu
    Copy the full SHA
    abc3d25 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2020

  1. PhoBERT - with Auto mode - without import fastBPE

    @datquocnguyen
    datquocnguyen committed Aug 30, 2020
    Configuration menu
    Copy the full SHA
    1287460 View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2020

  1. PhoBERT - with Auto mode - without import fastBPE

    @datquocnguyen
    datquocnguyen committed Aug 31, 2020
    Configuration menu
    Copy the full SHA
    efcbb59 View commit details
    Browse the repository at this point in the history

  2. BERTweet - with Auto mode - without import fastBPE

    @datquocnguyen
    datquocnguyen committed Aug 31, 2020
    Configuration menu
    Copy the full SHA
    1a10bb5 View commit details
    Browse the repository at this point in the history

  3. Add PhoBERT and BERTweet to TF modeling auto

    @datquocnguyen
    datquocnguyen committed Aug 31, 2020
    Configuration menu
    Copy the full SHA
    6c95a39 View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2020

  1. Improve Docstrings for PhobertTokenizer and BertweetTokenizer

    @datquocnguyen
    datquocnguyen committed Sep 1, 2020
    Configuration menu
    Copy the full SHA
    65ad9bb View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2020

  1. Update PhoBERT and BERTweet model cards

    @datquocnguyen
    datquocnguyen committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    6592499 View commit details
    Browse the repository at this point in the history

  2. Resolved merge conflicts

    @datquocnguyen
    datquocnguyen committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    80b5cb2 View commit details
    Browse the repository at this point in the history

  3. Fixed a merge conflict in tokenization_auto

    @datquocnguyen
    datquocnguyen committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    db78504 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2020

  1. Used black to reformat BERTweet- and PhoBERT-related files

    @datquocnguyen
    datquocnguyen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    384c599 View commit details
    Browse the repository at this point in the history

  2. Used isort to reformat BERTweet- and PhoBERT-related files

    @datquocnguyen
    datquocnguyen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    50407ac View commit details
    Browse the repository at this point in the history

  3. Reformatted BERTweet- and PhoBERT-related files based on flake8

    @datquocnguyen
    datquocnguyen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    28f514a View commit details
    Browse the repository at this point in the history

  4. Updated test files

    @datquocnguyen
    datquocnguyen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    bda755b View commit details
    Browse the repository at this point in the history

  5. Updated test files

    @datquocnguyen
    datquocnguyen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    8e5aa34 View commit details
    Browse the repository at this point in the history

  6. Updated tf test files

    @datquocnguyen
    datquocnguyen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    6d6e9cd View commit details
    Browse the repository at this point in the history

  7. Updated tf test files

    @datquocnguyen
    datquocnguyen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    f9d694c View commit details
    Browse the repository at this point in the history

  8. Updated tf test files

    @datquocnguyen
    datquocnguyen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    0811a1a View commit details
    Browse the repository at this point in the history

  9. Updated tf test files

    @datquocnguyen
    datquocnguyen committed Sep 4, 2020
    Configuration menu
    Copy the full SHA
    7e083b9 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2020

  1. Update commits from huggingface

    @datquocnguyen
    datquocnguyen committed Sep 15, 2020
    Configuration menu
    Copy the full SHA
    3885e12 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of git://github.com/huggingface/transformers

    @datquocnguyen
    datquocnguyen committed Sep 15, 2020
    Configuration menu
    Copy the full SHA
    926a6f6 View commit details
    Browse the repository at this point in the history

  3. Update commits from huggingface

    @datquocnguyen
    datquocnguyen committed Sep 15, 2020
    Configuration menu
    Copy the full SHA
    8db4b5b View commit details
    Browse the repository at this point in the history

  4. Delete unnecessary files

    @datquocnguyen
    datquocnguyen committed Sep 15, 2020
    Configuration menu
    Copy the full SHA
    3a883d2 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2020

  1. Add tokenizers to auto and init files

    @datquocnguyen
    datquocnguyen committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    e1ee98a View commit details
    Browse the repository at this point in the history

  2. Add test files for tokenizers

    @datquocnguyen
    datquocnguyen committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    f15b08b View commit details
    Browse the repository at this point in the history

  3. Revised model cards

    @datquocnguyen
    datquocnguyen committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    62beafe View commit details
    Browse the repository at this point in the history

  4. Update save_vocabulary function in BertweetTokenizer and PhobertToken… 

    …izer and test files
    @datquocnguyen
    datquocnguyen committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    73de519 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2020

  1. Revised test files

    @datquocnguyen
    datquocnguyen committed Sep 17, 2020
    Configuration menu
    Copy the full SHA
    1207489 View commit details
    Browse the repository at this point in the history

  2. Update orders of Phobert and Bertweet tokenizers in auto tokenization… 

    … file
    @datquocnguyen
    datquocnguyen committed Sep 17, 2020
    Configuration menu
    Copy the full SHA
    257b9f1 View commit details
    Browse the repository at this point in the history