Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🚨🚨🚨 [NLLB Tokenizer] Fix the prefix tokens 🚨🚨🚨 #22313

Merged
merged 10 commits into from Apr 4, 2023
Merged

🚨🚨🚨 [NLLB Tokenizer] Fix the prefix tokens 🚨🚨🚨 #22313

merged 10 commits into from Apr 4, 2023
+111 −23

Commits on Mar 22, 2023

  1. fix the prefix tokens

    @ArthurZucker
    ArthurZucker committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    79e6c83 View commit details
    Browse the repository at this point in the history

Commits on Mar 31, 2023

  1. update fast and test values

    @ArthurZucker
    ArthurZucker committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    f4e0eff View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/huggingface/transformers in… 

    …to fix-nllb-tokenizer
    @ArthurZucker
    ArthurZucker committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    5b3d111 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2023

  1. add legacy behaviour 

    Co-authored-by: sgugger <sylvain.gugger@gmail.com>
    @ArthurZucker @sgugger
    ArthurZucker and sgugger committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    152fcdf View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/huggingface/transformers in… 

    …to fix-nllb-tokenizer
    @ArthurZucker
    ArthurZucker committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    b0e9c03 View commit details
    Browse the repository at this point in the history

  3. update disclaimer, linkissue PR and behaviral changes

    @ArthurZucker
    ArthurZucker committed Apr 3, 2023
    Configuration menu
    Copy the full SHA
    f4c4c77 View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2023

  1. Apply suggestions from code review 

    Co-authored-by: Lysandre Debut <hi@lysand.re>
    @ArthurZucker @LysandreJik
    ArthurZucker and LysandreJik committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    550503e View commit details
    Browse the repository at this point in the history

  2. styling

    @ArthurZucker
    ArthurZucker committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    20df4e9 View commit details
    Browse the repository at this point in the history

  3. make a quote

    @ArthurZucker
    ArthurZucker committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    aa0fbb9 View commit details
    Browse the repository at this point in the history

  4. quote this time

    @ArthurZucker
    ArthurZucker committed Apr 4, 2023
    Configuration menu
    Copy the full SHA
    59c4209 View commit details
    Browse the repository at this point in the history