Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix fast tokenization problems #13930

Merged
merged 5 commits into from
Nov 10, 2021
Merged

Fix fast tokenization problems #13930

merged 5 commits into from
Nov 10, 2021

Commits on Oct 8, 2021

  1. Fix albert mask token tokenization.

    @qqaatw
    qqaatw committed Oct 8, 2021
    Configuration menu
    Copy the full SHA
    721f89a View commit details
    Browse the repository at this point in the history

  2. Ensure special tokans sanitized.

    @qqaatw
    qqaatw committed Oct 8, 2021
    Configuration menu
    Copy the full SHA
    2d1aafc View commit details
    Browse the repository at this point in the history

  3. Style

    @qqaatw
    qqaatw committed Oct 8, 2021
    Configuration menu
    Copy the full SHA
    6f2d501 View commit details
    Browse the repository at this point in the history

  4. Fix

    @qqaatw
    qqaatw committed Oct 8, 2021
    Configuration menu
    Copy the full SHA
    a72fab7 View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2021

  1. Apply suggestions from code review

    @qqaatw
    qqaatw committed Nov 10, 2021
    Configuration menu
    Copy the full SHA
    4b18163 View commit details
    Browse the repository at this point in the history