Add LayoutXLMProcessor (and LayoutXLMTokenizer, LayoutXLMTokenizerFast)#14115

Merged

NielsRogge merged 23 commits intohuggingface:masterfrom NielsRogge:add-layoutxlm-fast-tokenizer

Nov 3, 2021

+4,294-4

Commits on Nov 2, 2021

Add LayoutXLMTokenizer and LayoutXLMTokenizerFast

kingyiusuen
authored and
NielsRogge
committed
Fix styling issues

kingyiusuen
authored and
NielsRogge
committed
Fix more styling issues

kingyiusuen
authored and
NielsRogge
committed
Fix more styling issues

kingyiusuen
authored and
NielsRogge
committed
Fix docstring

kingyiusuen
authored and
NielsRogge
committed
Fix unit tests

kingyiusuen
authored and
NielsRogge
committed
Fix docs

kingyiusuen
authored and
NielsRogge
committed
Fix unit tests

kingyiusuen
authored and
NielsRogge
committed
Fix typos and styling issues

kingyiusuen
authored and
NielsRogge
committed
Fix styling issues

kingyiusuen
authored and
NielsRogge
committed
Fix docstring

kingyiusuen
authored and
NielsRogge
committed
Make all tests of test_tokenization_layoutxlm pass
NielsRogge
committed
Add LayoutXLMProcessor
NielsRogge
committed
Make fixup
NielsRogge
committed
Make all LayoutXLMProcessor tests pass
NielsRogge
committed
Minor fixes
NielsRogge
committed
Leave LayoutLMv2Processor tests unchanged
NielsRogge
committed
Fix code quality
NielsRogge
committed
Move LayoutXLM tokenizers and processor to separate folder
NielsRogge
committed
Fix code quality
NielsRogge
committed
Apply suggestions from code review
NielsRogge
committed
Replace assertions by value errors
NielsRogge
committed
Remove methods from fast tokenizer
NielsRogge
committed