Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add WordpieceTokenizer #72

Merged
merged 11 commits into from Mar 11, 2021
Merged

Add WordpieceTokenizer #72

merged 11 commits into from Mar 11, 2021

Commits on Mar 4, 2021

  1. Add BertTokenizer 

    Signed-off-by: xavier dupré <xavier.dupre@gmail.com>
    @sdpython
    sdpython committed Mar 4, 2021
    Configuration menu
    Copy the full SHA
    4a6997e View commit details
    Browse the repository at this point in the history

Commits on Mar 5, 2021

  1. add wordpiece tokenizer 

    Signed-off-by: xavier dupré <xavier.dupre@gmail.com>
    @sdpython
    sdpython committed Mar 5, 2021
    Configuration menu
    Copy the full SHA
    98c3e55 View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2021

  1. add string output 

    Signed-off-by: xavier dupré <xavier.dupre@gmail.com>
    @sdpython
    sdpython committed Mar 8, 2021
    Configuration menu
    Copy the full SHA
    73ab69e View commit details
    Browse the repository at this point in the history

  2. doc 

    Signed-off-by: xavier dupré <xavier.dupre@gmail.com>
    @sdpython
    sdpython committed Mar 8, 2021
    Configuration menu
    Copy the full SHA
    5dcb900 View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2021

  1. add RaggedTensorToDense 

    Signed-off-by: xavier dupré <xavier.dupre@gmail.com>
    @sdpython
    sdpython committed Mar 10, 2021
    Configuration menu
    Copy the full SHA
    459a159 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/microsoft/ort-customops int… 

    …o bert
    @sdpython
    sdpython committed Mar 10, 2021
    Configuration menu
    Copy the full SHA
    8dcc161 View commit details
    Browse the repository at this point in the history

  3. regex split 

    Signed-off-by: xavier dupré <xavier.dupre@gmail.com>
    @sdpython
    sdpython committed Mar 10, 2021
    Configuration menu
    Copy the full SHA
    5ddc5ea View commit details
    Browse the repository at this point in the history

Commits on Mar 11, 2021

  1. update documentation 

    Signed-off-by: xavier dupré <xavier.dupre@gmail.com>
    @sdpython
    sdpython committed Mar 11, 2021
    Configuration menu
    Copy the full SHA
    8c47dcd View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/microsoft/ort-customops int… 

    …o bert
    @sdpython
    sdpython committed Mar 11, 2021
    Configuration menu
    Copy the full SHA
    3083153 View commit details
    Browse the repository at this point in the history

  3. fix tokenizers 

    Signed-off-by: xavier dupré <xavier.dupre@gmail.com>
    @sdpython
    sdpython committed Mar 11, 2021
    Configuration menu
    Copy the full SHA
    cb49429 View commit details
    Browse the repository at this point in the history

  4. ragged 

    Signed-off-by: xavier dupré <xavier.dupre@gmail.com>
    @sdpython
    sdpython committed Mar 11, 2021
    Configuration menu
    Copy the full SHA
    8874602 View commit details
    Browse the repository at this point in the history