TrOCR small processors are all broken

[samwarren]

Environment info

• transformers version: 4.15.0
• Platform: macOS-10.15.7-x86_64-i386-64bit
• Python version: 3.9.7
• PyTorch version (GPU?): 1.10.0 (False)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: no
• Using distributed or parallel set-up in script?: no

Who can help

@LysandreJik

Information

Model I am using TrOCR

The problem arises when using:

• [ Y] the official example scripts: (give details below)
• my own modified scripts: (give details below)

The tasks I am working on is:

• an official GLUE/SQUaD task: (give the name)
• my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

1. Run the example at https://huggingface.co/microsoft/trocr-small-handwritten#how-to-use

Code:

from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image
import requests

# load image from the IAM database
url = 'https://fki.tic.heia-fr.ch/static/img/a01-122-02-00.jpg'
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")

processor = TrOCRProcessor.from_pretrained('microsoft/trocr-small-handwritten')
model = VisionEncoderDecoderModel.from_pretrained('microsoft/trocr-small-handwritten')
pixel_values = processor(images=image, return_tensors="pt").pixel_values

generated_ids = model.generate(pixel_values)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

Error trace:

The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'XLMRobertaTokenizer'. 
The class this function is called from is 'RobertaTokenizer'.
Traceback (most recent call last):
  File "/Users/samuel.warren/development/signature_detection/src/trocr_issue.py", line 9, in <module>
    processor = TrOCRProcessor.from_pretrained('microsoft/trocr-small-handwritten')
  File "/usr/local/Caskroom/miniconda/base/envs/sd39/lib/python3.9/site-packages/transformers/models/trocr/processing_trocr.py", line 109, in from_pretrained
    tokenizer = RobertaTokenizer.from_pretrained(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/Caskroom/miniconda/base/envs/sd39/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1747, in from_pretrained
    return cls._from_pretrained(
  File "/usr/local/Caskroom/miniconda/base/envs/sd39/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1882, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/usr/local/Caskroom/miniconda/base/envs/sd39/lib/python3.9/site-packages/transformers/models/roberta/tokenization_roberta.py", line 166, in __init__
    super().__init__(
  File "/usr/local/Caskroom/miniconda/base/envs/sd39/lib/python3.9/site-packages/transformers/models/gpt2/tokenization_gpt2.py", line 180, in __init__
    with open(vocab_file, encoding="utf-8") as vocab_handle:
TypeError: expected str, bytes or os.PathLike object, not NoneType

Expected behavior

I would expect it to run and output the text.

[dan-locke]

See Issue #14893. You'll have to pip install from the GitHub source: pip install transformers git+https://github.com/huggingface/transformers.git to get working small trocr transformers.

[samwarren]

working now, thank you!

[NielsRogge]

Closing, since this has been fixed (the small TrOCR models are only supported since the PR linked above, which will be included in the next release).