Failed to run main_demo_7B.py

[notus126]

I try to run the main_demo_7B.py. The log of the error is as follows:

[2023-12-14 13:14:17,810] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Traceback (most recent call last):
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/urllib3/connection.py", line 203, in _new_conn
    sock = connection.create_connection(
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1096, in _validate_conn
    conn.connect()
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/urllib3/connection.py", line 611, in connect
    self.sock = sock = self._new_conn()
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/urllib3/connection.py", line 210, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7f2b949caf50>: Failed to resolve 'huggingface.co' ([Errno -3] Temporary failure in name resolution)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/urllib3/connectionpool.py", line 844, in urlopen
    retries = retries.increment(
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /openai/clip-vit-large-patch14/resolve/main/config.json (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f2b949caf50>: Failed to resolve 'huggingface.co' ([Errno -3] Temporary failure in name resolution)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1247, in hf_hub_download
    metadata = get_hf_file_metadata(
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1624, in get_hf_file_metadata
    r = _request_wrapper(
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 402, in _request_wrapper
    response = _request_wrapper(
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 425, in _request_wrapper
    response = get_session().request(method=method, url=url, **params)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 63, in send
    return super().send(request, *args, **kwargs)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: (MaxRetryError('HTTPSConnectionPool(host=\'huggingface.co\', port=443): Max retries exceeded with url: /openai/clip-vit-large-patch14/resolve/main/config.json (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f2b949caf50>: Failed to resolve \'huggingface.co\' ([Errno -3] Temporary failure in name resolution)"))'), '(Request ID: a26cb2d6-252e-4c54-bfdd-9c5a0dfe2a1b)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/transformers/utils/hub.py", line 417, in cached_file
    resolved_file = hf_hub_download(
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1377, in hf_hub_download
    raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ning/miniconda3/envs/chatunivi/bin/uvicorn", line 8, in <module>
    sys.exit(main())
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/uvicorn/main.py", line 416, in main
    run(
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/uvicorn/main.py", line 587, in run
    server.run()
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/uvicorn/server.py", line 61, in run
    return asyncio.run(self.serve(sockets=sockets))
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/uvicorn/server.py", line 68, in serve
    config.load()
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/uvicorn/config.py", line 467, in load
    self.loaded_app = import_from_string(self.app)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string
    module = importlib.import_module(module_str)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ning/Chat-UniVi/main_demo_7B.py", line 121, in <module>
    handler = Chat(model_path, conv_mode=conv_mode)
  File "/home/ning/Chat-UniVi/ChatUniVi/demo.py", line 16, in __init__
    self.tokenizer, self.model, self.image_processor, context_len = load_pretrained_model(model_path, None, model_name="ChatUniVi")
  File "/home/ning/Chat-UniVi/ChatUniVi/model/builder.py", line 75, in load_pretrained_model
    model = AutoModelForCausalLM.from_pretrained(model_path, low_cpu_mem_usage=True, **kwargs)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 493, in from_pretrained
    return model_class.from_pretrained(
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2700, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "/home/ning/Chat-UniVi/ChatUniVi/model/language_model/llama.py", line 28, in __init__
    self.model = ChatUniViLlamaModel(config)
  File "/home/ning/Chat-UniVi/ChatUniVi/model/language_model/llama.py", line 20, in __init__
    super(ChatUniViLlamaModel, self).__init__(config)
  File "/home/ning/Chat-UniVi/ChatUniVi/model/arch.py", line 15, in __init__
    self.vision_tower = build_vision_tower(config, delay_load=True)
  File "/home/ning/Chat-UniVi/ChatUniVi/model/multimodal_encoder/builder.py", line 8, in build_vision_tower
    return CLIPVisionTower(vision_tower, args=vision_tower_cfg, **kwargs)
  File "/home/ning/Chat-UniVi/ChatUniVi/model/multimodal_encoder/clip_encoder.py", line 24, in __init__
    self.cfg_only = CLIPVisionConfig.from_pretrained(self.vision_tower_name)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/transformers/models/clip/configuration_clip.py", line 239, in from_pretrained
    config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/transformers/configuration_utils.py", line 618, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/transformers/configuration_utils.py", line 673, in _get_config_dict
    resolved_config_file = cached_file(
  File "/home/ning/miniconda3/envs/chatunivi/lib/python3.10/site-packages/transformers/utils/hub.py", line 452, in cached_file
    raise EnvironmentError(
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like openai/clip-vit-large-patch14 is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.

I guess the error is caused by the disconnection to huggingface, and failure to download weights of clip-vit-large-patch14. I want to know where is the path of clip-vit-large-patch14, so I can modify it to locate my local model weights. Anyone can help me?

[jpthu17]

If you're unable to connect to Hugging Face, you can manually download the weights for clip-vit-large-patch14.

For instance, you can locally download the weights via a mirror site:

pip install -U huggingface_hub
export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download --resume-download --local-dir-use-symlinks False openai/clip-vit-large-patch14 --local-dir ${save_path}

Next, to ensure proper loading of the model, you'll need to set this parameter to the address where you've stored the CLIP weights:

vision_tower = ${save_path}
return CLIPVisionTower(vision_tower, args=vision_tower_cfg, **kwargs)