Add LayoutXLMTokenizer and LayoutXLMTokenizerFast

[kingyiusuen]

What does this PR do?

Fixes #13972 (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[NielsRogge]

Suggested change

	:class:`LayoutXLMTokenizer` or :class:`LayoutXLMTokenizerFast`which turns the words and bounding boxes into
	:class:`LayoutXLMTokenizer` or :class:`LayoutXLMTokenizerFast` which turn the words and bounding boxes into

[NielsRogge]

Suggested change

	# Copyright 2018 Google AI, Google Brain and Carnegie Mellon University Authors and the HuggingFace Inc. team.
	# Copyright 2021 the HuggingFace Inc. team.

[NielsRogge]

Suggested change

	# Copyright 2018 Google AI, Google Brain and Carnegie Mellon University Authors and the HuggingFace Inc. team.
	# Copyright 2021 The HuggingFace Inc. team.

[NielsRogge]

Nice! Let me know if you need any help.

[kingyiusuen]

Nice! Let me know if you need any help.

Thanks!

The tokenizer in 'microsoft/layoutxlm-base' was registered with XLMRobertaTokenizer, so the user will get a warning that there is a mismatch when they do

tokenizer = LayoutXLMTokenizer.from_pretrained('microsoft/layoutxlm-base') 

Is there a way to avoid this?

Also, I keep getting a ModuleNotFoundError: No module named 'sentencepiece' error in tests/test_processor_layoutlmv2.py. I can't figure out what is wrong.

[NielsRogge]

The tokenizer in 'microsoft/layoutxlm-base' was registered with XLMRobertaTokenizer, so the user will get a warning

Yeah, that's because the config currently has an attribute tokenizer_class which is set to XLMRobertaTokenizer as can be seen here. Once LayoutXLMTokenizer/LayoutXLMTokenizerFast are ready, we will upload the vocab files to the model repo, and remove the tokenizer_class attribute.

[kingyiusuen]

The tokenizer in 'microsoft/layoutxlm-base' was registered with XLMRobertaTokenizer, so the user will get a warning

Yeah, that's because the config currently has an attribute tokenizer_class which is set to XLMRobertaTokenizer as can be seen here. Once LayoutXLMTokenizer/LayoutXLMTokenizerFast are ready, we will upload the vocab files to the model repo, and remove the tokenizer_class attribute.

Got it. I just have a failing test and a docstring error in the CI. Any advice on how to fix them?

[NielsRogge]

Ok, I'll take a look later today.

[NielsRogge]

@kingyiusuen I've fixed the docs issue, currently checking out the tests. One is failing, the boxes created between the slow and fast tokenizer aren't equal:

from transformers import LayoutXLMTokenizer, LayoutXLMTokenizerFast

tokenizer_p = LayoutXLMTokenizer.from_pretrained("microsoft/layoutxlm-base")
tokenizer_r = LayoutXLMTokenizerFast.from_pretrained("microsoft/layoutxlm-base")
question = "what's his name?"
words = ["a", "weirdly", "test"]
boxes = [[423, 237, 440, 251], [427, 272, 441, 287], [419, 115, 437, 129]]

encoding_p = tokenizer_p(question, words, boxes, padding="max_length", max_length=20)
encoding_r = tokenizer_r(question, words, boxes, padding="max_length", max_length=20)
for x,y in zip(encoding_p.bbox, encoding_r.bbox): print(x,y)

# this prints:
[0, 0, 0, 0] [0, 0, 0, 0]
[0, 0, 0, 0] [0, 0, 0, 0]
[0, 0, 0, 0] [0, 0, 0, 0]
[0, 0, 0, 0] [0, 0, 0, 0]
[0, 0, 0, 0] [0, 0, 0, 0]
[0, 0, 0, 0] [0, 0, 0, 0]
[0, 0, 0, 0] [0, 0, 0, 0]
[1000, 1000, 1000, 1000] [1000, 1000, 1000, 1000]
[423, 237, 440, 251] [1000, 1000, 1000, 1000]
[427, 272, 441, 287] [423, 237, 440, 251]
[427, 272, 441, 287] [427, 272, 441, 287]
[419, 115, 437, 129] [427, 272, 441, 287]
[1000, 1000, 1000, 1000] [419, 115, 437, 129]
[0, 0, 0, 0] [1000, 1000, 1000, 1000]
[0, 0, 0, 0] [0, 0, 0, 0]
[0, 0, 0, 0] [0, 0, 0, 0]
[0, 0, 0, 0] [0, 0, 0, 0]
[0, 0, 0, 0] [0, 0, 0, 0]
[0, 0, 0, 0] [0, 0, 0, 0]

The input_ids and attention_mask are equal. Decoding the input_ids, it seems that one adds 2 special tokens in between the question and the context? So the fast tokenizer seems correct.

UPDATE: fixed it in the slow tokenizer.

[NielsRogge]

It might make sense to create a separate LayoutXLMProcessor. Will do this.

[NielsRogge]

I've made all necessary changes, fixed all tests, implemented a new LayoutXLMProcessor and created new tests for it accordingly.

You can find my branch here: https://github.com/NielsRogge/transformers/tree/add-layoutxlm-fast-tokenizer

Should I open a PR on your branch? Or should I directly open a PR to HuggingFace Transformers?

[kingyiusuen]

I've made all necessary changes, fixed all tests, implemented a new LayoutXLMProcessor and created new tests for it accordingly.

You can find my branch here: https://github.com/NielsRogge/transformers/tree/add-layoutxlm-fast-tokenizer

Should I open a PR on your branch? Or should I directly open a PR to HuggingFace Transformers?

Maybe you should directly open a PR to HuggingFace Transformers. You've done most of the heavy-lifting. This should be counted as your contribution. 😃