IsADirectoryError: [Errno 21] Is a directory: 'checkpoints/clip-vit-large-patch14'

[mondalanindya]

Thanks for your amazing work. I was trying to download the ViT-L checkpoints from HF by cloning the whole repository. However, I am getting the error shown above. Can you suggest a fix for this?

[jefferyZhan]

Hi @mondalanindya ,
could you please provide us with more details about raising error?

[mondalanindya]

Here is the full text:

Traceback (most recent call last):
  File "/home/amondal/Codes/Griffon/tools/resize_clip_pos.py", line 68, in <module>
    state_dict = copy.deepcopy(torch.load(args.model_path, map_location="cpu"))
  File "/home/amondal/anaconda3/envs/awesome_env2.0/lib/python3.10/site-packages/torch/serialization.py", line 791, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/home/amondal/anaconda3/envs/awesome_env2.0/lib/python3.10/site-packages/torch/serialization.py", line 271, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/home/amondal/anaconda3/envs/awesome_env2.0/lib/python3.10/site-packages/torch/serialization.py", line 252, in __init__
    super().__init__(open(name, mode))
IsADirectoryError: [Errno 21] Is a directory: 'checkpoints/clip-vit-large-patch14'

Probably it expects a file instead of a directory containing the ViT-L checkpoint.

[dawn-ech]

@mondalanindya I solve this by replacing 'checkpoints/clip-vit-large-patch14' with 'checkpoints/clip-vit-large-patch14/pytorch_model.bin', and 'checkpoints/clip-vit-large-patch14-448' with 'checkpoints/clip-vit-large-patch14-448/pytorch_model.bin'

[jefferyZhan]

This is due to the version error. We fixed it with automatic search. Thanks a lot. @mondalanindya @dawn-ech

[mondalanindya]

Thanks for the changes! I am getting another error even after running the modified script

bash demo/demo.sh demo/1v1.jpg "Is there a motorcycle on the far left of the photo?" demo/1v1.jpg Is there a motorcycle on the far left of the photo? CUDA_VISIBLE_DEVICES=0 python llava/eval/run_llava.py --model-path checkpoints/clip-vit-large-patch14 checkpoints/clip-vit-large-patch14-448 checkpoints/clip-vit-large-patch14_to_448 checkpoints/Griffon --image-file demo/1v1.jpg --query Is there a motorcycle on the far left of the photo? --obj target [2024-05-07 11:25:25,801] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect) usage: run_llava.py [-h] [--model-path MODEL_PATH] [--model-base MODEL_BASE] --image-file IMAGE_FILE --query QUERY --obj OBJ run_llava.py: error: unrecognized arguments: checkpoints/clip-vit-large-patch14-448 checkpoints/clip-vit-large-patch14_to_448 checkpoints/Griffon

[jefferyZhan]

Please refer to the args in the demo.* file. It seems you input the unused arguments "checkpoints/clip-vit-large-patch14-448 checkpoints/clip-vit-large-patch14_to_448 checkpoints/Griffon".

[mondalanindya]

Hi, there are three different checkpoints:
checkpoints/clip-vit-large-patch14-448 checkpoints/clip-vit-large-patch14_to_448 checkpoints/Griffon

which one should I use?

[JierunChen]

@jefferyZhan Hi, could you be more specific and exact in how to run the demo.sh? It would be great if you could give the step by step guidelines.