Invalid load key error when running models_server.py

[panicbuttonvfx]

When running:

python models_server.py --config config.yaml

I get the following error:

Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/jarvis/lib/python3.8/site-packages/transformers/modeling_utils.py", line 415, in load_state_dict
    return torch.load(checkpoint_file, map_location="cpu")
  File "/home/ubuntu/miniconda3/envs/jarvis/lib/python3.8/site-packages/torch/serialization.py", line 795, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/home/ubuntu/miniconda3/envs/jarvis/lib/python3.8/site-packages/torch/serialization.py", line 1002, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

[ErikDombi]

Is there any more to the traceback? I had a similar issue come up, but it was a far longer error.

[panicbuttonvfx]

There is, here's the full traceback. git-lfs is installed, initialized and updated:

Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/jarvis/lib/python3.8/site-packages/transformers/modeling_utils.py", line 415, in load_state_dict
    return torch.load(checkpoint_file, map_location="cpu")
  File "/home/ubuntu/miniconda3/envs/jarvis/lib/python3.8/site-packages/torch/serialization.py", line 795, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/home/ubuntu/miniconda3/envs/jarvis/lib/python3.8/site-packages/torch/serialization.py", line 1002, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "models_server.py", line 342, in <module>
    pipes = load_pipes(local_deployment)
  File "models_server.py", line 87, in load_pipes
    "model": VisionEncoderDecoderModel.from_pretrained(f"{local_fold}/nlpconnect/vit-gpt2-image-captioning"),
  File "/home/ubuntu/miniconda3/envs/jarvis/lib/python3.8/site-packages/transformers/models/vision_encoder_decoder/modeling_vision_encoder_decoder.py", line 363, in from_pretrained
    return super().from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
  File "/home/ubuntu/miniconda3/envs/jarvis/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2477, in from_pretrained
    state_dict = load_state_dict(resolved_archive_file)
  File "/home/ubuntu/miniconda3/envs/jarvis/lib/python3.8/site-packages/transformers/modeling_utils.py", line 420, in load_state_dict
    raise OSError(
OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run `git lfs install` followed by `git lfs pull` in the folder you cloned.

[ErikDombi]

If you installed git-lfs AFTER downloading the models, then you will need to rerun the download.sh script inside jarvis/server/models/

[panicbuttonvfx]

Thanks, that seems to be what was happening. Download hasn't completed yet though (it keeps getting stuck at some point), so I haven't been able to check if it works. I'll close this issue once I'm able to test it out.

[panicbuttonvfx]

I'm closing this issue, solved after re-download the models.