Code

Add BlenderbotTokenizerFast#13720

Code

Merged

LysandreJik merged 19 commits intohuggingface:masterhuggingface/transformers:masterfrom stancld:blenderbot_fast_tokenizerstancld/transformers:blenderbot_fast_tokenizerCopy head branch name to clipboardOct 29, 2021

+181-6

Commits

Commits on Sep 20, 2021

• Add the support for the fast (rust) implementation of BlenbderbotTokenizer

  stancld

  committed
  a02b805

  Copy full SHA for a02b805

Commits on Sep 23, 2021

• Merge branch 'master' into blenderbot_fast_tokenizer

  stancld

  committed
  df77310

  Copy full SHA for df77310

• Fix a converter and a typo in a doc

  stancld

  committed
  4d8702c

  Copy full SHA for 4d8702c

• Merge branch 'master' into blenderbot_fast_tokenizer

  stancld

  committed
  53f36ba

  Copy full SHA for 53f36ba

Commits on Sep 25, 2021

• Merge branch 'master' into blenderbot_fast_tokenizer

  stancld

  committed
  f3cc692

  Copy full SHA for f3cc692

Commits on Sep 29, 2021

• Merge branch 'master' into blenderbot_fast_tokenizer

  stancld

  committed
  b27b247

  Copy full SHA for b27b247

• Apply the patil-suraj's suggestion

  stancld

  committed
  3a9cb48

  Copy full SHA for 3a9cb48

• (Nitpick) Fast tokenization -> Fast Tokenization in doc

  stancld

  committed
  12d6ae0

  Copy full SHA for 12d6ae0

Commits on Sep 30, 2021

• Merge branch 'master' into blenderbot_fast_tokenizer

  stancld

  committed
  4d78b39

  Copy full SHA for 4d78b39

• Apply the SaulLu's suggestion

  stancld

  committed
  f24e104

  Copy full SHA for f24e104

Commits on Oct 15, 2021

• Merge branch 'master' into blenderbot_fast_tokenizer

  stancld

  committed
  918bf93

  Copy full SHA for 918bf93

• Apply Narsil's suggestion to fix test pipelines

  stancld

  committed
  025432b

  Copy full SHA for 025432b

• Add encoder_no_repeat_ngram_size according to the Narsil's suggestion

  stancld

  committed
  d3d4b1a

  Copy full SHA for d3d4b1a

• Revert the last (unnecessary) commit

  stancld

  committed
  aa0cb70

  Copy full SHA for aa0cb70

Commits on Oct 20, 2021

• Merge branch 'master' into blenderbot_fast_tokenizer

  stancld

  committed
  423fe52

  Copy full SHA for 423fe52

• Override pipeline config for Blenderbot to allow for larger pos. emb.

  stancld

  committed
  be988fb

  Copy full SHA for be988fb

Commits on Oct 22, 2021

• Merge branch 'master' into blenderbot_fast_tokenizer

  stancld

  committed
  ee3c72e

  Copy full SHA for ee3c72e

• make fix-copies

  stancld

  committed
  a393420

  Copy full SHA for a393420

Commits on Oct 29, 2021

• Merge branch 'master' into blenderbot_fast_tokenizer

  stancld

  committed
  a40b52d

  Copy full SHA for a40b52d