[Usage] ImportError: cannot import name 'LlavaLlamaForCausalLM' from 'llava.model' #1101

TobiasJu · 2024-02-08T02:02:37Z

Describe the issue

Issue:
So i updated the repo and now i can not start the server anymore. I deleted the repo and cloned it again, but get the same error.

System:
Win10 WSL2 Ubuntu
pip 24.0 from /home/tobias/.local/lib/python3.10/site-packages/pip (python 3.10)

Command:

git clone https://github.com/haotian-liu/LLaVA.git
cd LLaVA
conda create --name LLaVA python=3.10.12
conda activate LLaV
pip install -e .
pip install flash-attn --no-build-isolation
python3 -m llava.serve.controller --host 0.0.0.0 --port 10000

Log:

python3 -m llava.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --model-path liuhaotian/llava-v1.5-13b --load-4bit
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.10/runpy.py", line 110, in _get_module_details
    __import__(pkg_name)
  File "/mnt/a/KI/LLaVA/llava/__init__.py", line 1, in <module>
    from .model import LlavaLlamaForCausalLM
ImportError: cannot import name 'LlavaLlamaForCausalLM' from 'llava.model' (/mnt/a/KI/LLaVA/llava/model/__init__.py)

Packages:

pip list
Package                   Version       Editable project location
------------------------- ------------- -------------------------
accelerate                0.21.0
aiofiles                  23.2.1
aiohttp                   3.9.3
aiosignal                 1.3.1
altair                    5.2.0
annotated-types           0.6.0
anyio                     4.2.0
async-timeout             4.0.3
attrs                     23.2.0
bitsandbytes              0.41.0
blinker                   1.4
CacheControl              0.12.10
cachy                     0.3.0
certifi                   2023.11.17
chardet                   4.0.0
charset-normalizer        3.3.2
cleo                      0.8.1
click                     8.1.7
clikit                    0.6.2
cmake                     3.28.1
colorama                  0.4.6
command-not-found         0.3
contourpy                 1.2.0
crashtest                 0.3.1
cryptography              3.4.8
cycler                    0.12.1
dbus-python               1.2.18
distlib                   0.3.4
distro                    1.7.0
distro-info               1.1+ubuntu0.2
einops                    0.6.1
einops-exts               0.0.4
exceptiongroup            1.2.0
fastapi                   0.109.0
ffmpy                     0.3.1
filelock                  3.13.1
flash-attn                2.5.2
fonttools                 4.47.2
frozenlist                1.4.1
fsspec                    2023.12.2
gradio                    4.16.0
gradio_client             0.8.1
h11                       0.14.0
html5lib                  1.1
httpcore                  0.17.3
httplib2                  0.20.2
httpx                     0.24.0
huggingface-hub           0.20.3
idna                      3.6
importlib-metadata        4.6.4
importlib-resources       6.1.1
jeepney                   0.7.1
Jinja2                    3.1.3
joblib                    1.3.2
jsonschema                4.21.1
jsonschema-specifications 2023.12.1
keyring                   23.5.0
kiwisolver                1.4.5
launchpadlib              1.10.16
lazr.restfulclient        0.14.4
lazr.uri                  1.0.6
linkify-it-py             2.0.2
lit                       17.0.6
llava                     1.2.2.post1   /mnt/a/KI/LLaVA
lockfile                  0.12.2
markdown-it-py            2.2.0
markdown2                 2.4.12
MarkupSafe                2.1.4
matplotlib                3.8.2
mdit-py-plugins           0.3.3
mdurl                     0.1.2
more-itertools            8.10.0
mpmath                    1.3.0
msgpack                   1.0.3
multidict                 6.0.4
netifaces                 0.11.0
networkx                  3.2.1
ninja                     1.11.1.1
numpy                     1.26.3
nvidia-cublas-cu11        11.10.3.66
nvidia-cublas-cu12        12.1.3.1
nvidia-cuda-cupti-cu11    11.7.101
nvidia-cuda-cupti-cu12    12.1.105
nvidia-cuda-nvrtc-cu11    11.7.99
nvidia-cuda-nvrtc-cu12    12.1.105
nvidia-cuda-runtime-cu11  11.7.99
nvidia-cuda-runtime-cu12  12.1.105
nvidia-cudnn-cu11         8.5.0.96
nvidia-cudnn-cu12         8.9.2.26
nvidia-cufft-cu11         10.9.0.58
nvidia-cufft-cu12         11.0.2.54
nvidia-curand-cu11        10.2.10.91
nvidia-curand-cu12        10.3.2.106
nvidia-cusolver-cu11      11.4.0.1
nvidia-cusolver-cu12      11.4.5.107
nvidia-cusparse-cu11      11.7.4.91
nvidia-cusparse-cu12      12.1.0.106
nvidia-nccl-cu11          2.14.3
nvidia-nccl-cu12          2.18.1
nvidia-nvjitlink-cu12     12.3.101
nvidia-nvtx-cu11          11.7.91
nvidia-nvtx-cu12          12.1.105
oauthlib                  3.2.0
orjson                    3.9.12
packaging                 23.2
pandas                    2.2.0
pastel                    0.2.1
peft                      0.4.0
pexpect                   4.8.0
pillow                    10.2.0
pip                       24.0
pkginfo                   1.8.2
platformdirs              2.5.1
poetry-core               1.0.7
psutil                    5.9.8
ptyprocess                0.7.0
pydantic                  2.6.1
pydantic_core             2.16.2
pydub                     0.25.1
Pygments                  2.17.2
PyGObject                 3.42.1
PyJWT                     2.3.0
pylev                     1.2.0
pyparsing                 2.4.7
python-apt                2.4.0+ubuntu2
python-dateutil           2.8.2
python-multipart          0.0.6
pytz                      2023.4
PyYAML                    5.4.1
referencing               0.33.0
regex                     2023.12.25
requests                  2.31.0
requests-toolbelt         0.9.1
rich                      13.7.0
rpds-py                   0.17.1
ruff                      0.2.1
safetensors               0.4.2
scikit-learn              1.2.2
scipy                     1.12.0
SecretStorage             3.3.1
semantic-version          2.10.0
sentencepiece             0.1.99
setuptools                59.6.0
shellingham               1.4.0
shortuuid                 1.0.11
six                       1.16.0
sniffio                   1.3.0
starlette                 0.35.1
svgwrite                  1.4.3
sympy                     1.12
systemd-python            234
threadpoolctl             3.2.0
timm                      0.6.13
tokenizers                0.15.1
tomlkit                   0.12.0
toolz                     0.12.1
torch                     2.1.2
torchvision               0.16.2
tqdm                      4.66.1
transformers              4.37.2
triton                    2.1.0
typer                     0.9.0
typing_extensions         4.9.0
tzdata                    2023.4
ubuntu-advantage-tools    8001
uc-micro-py               1.0.2
ufw                       0.36.1
unattended-upgrades       0.1
urllib3                   2.2.0
uvicorn                   0.27.0.post1
virtualenv                20.13.0+ds
wadllib                   1.3.6
wavedrom                  2.0.3.post3
webencodings              0.5.1
websockets                11.0.3
wheel                     0.37.1
yarl                      1.9.4
zipp                      1.0.0

Screenshots:

The text was updated successfully, but these errors were encountered:

anubhavashok · 2024-02-08T05:25:43Z

Try importing the packages without the "try, except" block for a more informative error.

Probably related to flash attn installation.

In my case, the following worked:

pip uninstall flash-attn

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

pip install flash-attn --no-build-isolation --no-cache-dir

pseudotensor · 2024-02-08T09:42:56Z

For me it was problem with packages like deepspeed etc. that led to above. But any issue will lead to it. This is the combo that worked for me:

# CUDA_HOME may not be enough:
cd /usr/local/
sudo rm -rf /usr/local/cuda # if link only
sudo ln -s /usr/local/cuda-12.1 cuda

export CUDA_HOME=/usr/local/cuda-12.1
export PIP_EXTRA_INDEX_URL="https://download.pytorch.org/whl/cu121"

conda create -n llava python=3.10 -y
conda activate llava
pip install --upgrade pip  # enable PEP 660 support

git clone https://github.com/haotian-liu/LLaVA.git llava
cd llava

pip install -e .
pip install -e ".[train]"
pip install torch==2.1.2 torchvision==0.16.2 triton==2.1.0 accelerate==0.26.1 deepspeed==0.13.1 pynvml==11.5.0 --upgrade
pip install "sglang[all]"
pip install flash-attn==2.5.2 --no-build-isolation

celll1 · 2024-02-08T11:40:03Z

I am also having the same error.
I have found that this error occurs depending on the order in which the packages are installed, but have not been able to determine the cause.
If I create a new environment and then install only LLaVA, it works correctly, but I would like to use it in combination with other modules, so I would like to see this bug fixed.

TobiasJu · 2024-02-08T13:53:51Z

I got rid of the try-except block in llava/model/__init__.py and then reinstalled from root with pip install -e . and the error went away for me

before:

try:
    from .language_model.llava_llama import LlavaLlamaForCausalLM, LlavaConfig
    from .language_model.llava_mpt import LlavaMptForCausalLM, LlavaMptConfig
    from .language_model.llava_mistral import LlavaMistralForCausalLM, LlavaMistralConfig
except:
    pass

after:

from .language_model.llava_llama import LlavaLlamaForCausalLM, LlavaConfig
from .language_model.llava_mpt import LlavaMptForCausalLM, LlavaMptConfig
from .language_model.llava_mistral import LlavaMistralForCausalLM, LlavaMistralConfig

Tried and now i get this error:

python3 -m llava.serve.controller --host 0.0.0.0 --port 10000
[2024-02-08 14:52:59,501] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)
/home/tobias/.local/lib/python3.10/site-packages/pydantic/_internal/_config.py:322: UserWarning: Valid config keys have changed in V2:
* 'allow_population_by_field_name' has been renamed to 'populate_by_name'
* 'validate_all' has been renamed to 'validate_default'
  warnings.warn(message, UserWarning)
/home/tobias/.local/lib/python3.10/site-packages/pydantic/_internal/_fields.py:151: UserWarning: Field "model_persistence_threshold" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
Traceback (most recent call last):
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1364, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/generation/utils.py", line 28, in <module>
    from ..integrations.deepspeed import is_deepspeed_zero3_enabled
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/integrations/deepspeed.py", line 49, in <module>
    from accelerate.utils.deepspeed import HfDeepSpeedConfig as DeepSpeedConfig
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/__init__.py", line 3, in <module>
    from .accelerator import Accelerator
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/accelerator.py", line 35, in <module>
    from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/checkpointing.py", line 24, in <module>
    from .utils import (
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/utils/__init__.py", line 133, in <module>
    from .launch import (
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/utils/launch.py", line 33, in <module>
    from ..utils.other import is_port_in_use, merge_dicts
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/utils/other.py", line 30, in <module>
    from deepspeed import DeepSpeedEngine
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/__init__.py", line 16, in <module>
    from . import module_inject
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/module_inject/__init__.py", line 6, in <module>
    from .replace_module import replace_transformer_layer, revert_transformer_layer, ReplaceWithTensorSlicing, GroupQuantizer, generic_injection
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py", line 792, in <module>
    from ..pipe import PipelineModule
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/pipe/__init__.py", line 6, in <module>
    from ..runtime.pipe import PipelineModule, LayerSpec, TiedLayerSpec
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/pipe/__init__.py", line 6, in <module>
    from .module import PipelineModule, LayerSpec, TiedLayerSpec
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/pipe/module.py", line 19, in <module>
    from ..activation_checkpointing import checkpointing
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/activation_checkpointing/checkpointing.py", line 25, in <module>
    from deepspeed.runtime.config import DeepSpeedConfig
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/config.py", line 29, in <module>
    from .zero.config import get_zero_config, ZeroStageEnum
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/zero/__init__.py", line 6, in <module>
    from .partition_parameters import ZeroParamType
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/zero/partition_parameters.py", line 616, in <module>
    class Init(InsertPostInitMethodToModuleSubClasses):
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/zero/partition_parameters.py", line 618, in Init
    param_persistence_threshold = get_config_default(DeepSpeedZeroConfig, "param_persistence_threshold")
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/config_utils.py", line 116, in get_config_default
    field_name).required, f"'{field_name}' is a required field and does not have a default value"
AttributeError: 'FieldInfo' object has no attribute 'required'. Did you mean: 'is_required'?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1364, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 40, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 44, in <module>
    from .generation import GenerationConfig, GenerationMixin
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1354, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1366, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
'FieldInfo' object has no attribute 'required'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.10/runpy.py", line 110, in _get_module_details
    __import__(pkg_name)
  File "/mnt/a/KI/LLaVA/llava/__init__.py", line 1, in <module>
    from .model import LlavaLlamaForCausalLM
  File "/mnt/a/KI/LLaVA/llava/model/__init__.py", line 2, in <module>
    from .language_model.llava_llama import LlavaLlamaForCausalLM, LlavaConfig
  File "/mnt/a/KI/LLaVA/llava/model/language_model/llava_llama.py", line 21, in <module>
    from transformers import AutoConfig, AutoModelForCausalLM, \
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1355, in __getattr__
    value = getattr(module, name)
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1354, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1366, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
'FieldInfo' object has no attribute 'required'

pseudotensor · 2024-02-08T15:49:58Z

@TobiasJu I got same, but then did what I did:

#1101 (comment)

haotian-liu · 2024-02-08T16:29:44Z

We find that this is due to flash-attn compiled previously with a different version of pytorch. Please reinstall that with:

pip install flash-attn --no-build-isolation --no-cache-dir

pseudotensor · 2024-02-08T16:31:40Z

Correct, but even that isn't enough, deepspeed etc. have to be correct range of versions, else hit the other issues shown above.

haotian-liu · 2024-02-08T16:34:06Z

Thanks, deepspeed==0.12.6 works for me in my local setting, so basically I found that if you need to train the model, running the below command is sufficient?

pip install -e ".[train]"
pip install flash-attn --no-build-isolation --no-cache-dir

pseudotensor · 2024-02-08T16:40:49Z

In my and the above case the deepspeed error was hit with inference too.

I didn't check exactly which deepspeed version was required. I just did pip install accelerate deepspeed --upgrade and it started working, so I wanted to remember the versions that worked so wrote those down.

haotian-liu · 2024-02-08T16:48:55Z

Great, thank you for the information!

nitzanguetta · 2024-02-08T23:18:42Z

After following the instructions in the repo, I encountered the same error:

from .model import LlavaLlamaForCausalLM
ImportError: cannot import name 'LlavaLlamaForCausalLM' from 'llava.model' (/mnt/a/KI/LLaVA/llava/model/__init__.py)

Currently, after attempting #1101 (comment), I got this one:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/yonatanbitton/experiments/llava/llava/model/__init__.py", line 8, in <module>
    from .language_model.llava_llama import LlavaLlamaForCausalLM, LlavaConfig
  File "/home/yonatanbitton/experiments/llava/llava/model/language_model/llava_llama.py", line 21, in <module>
    from transformers import AutoConfig, AutoModelForCausalLM, \
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/opt/conda/envs/llava/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1355, in __getattr__
    value = getattr(module, name)
  File "/opt/conda/envs/llava/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1354, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/opt/conda/envs/llava/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1366, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):
libcudart.so.11.0: cannot open shared object file: No such file or directory

All attempts to resolve this issue have only resulted in more errors.

[ Debian GNU/Linux 11 (bullseye)
pip 24.0, python 3.10
(deepspeed==0.13.1/0.12.6, cuda==12.1) ]

pseudotensor · 2024-02-09T00:14:30Z

I got that libcudart.so.12.0: cannot open shared object file: No such file or directory type errors, but that's just because you must not have set CUDA_HOME corretly or do not have cuda toolkit installed for (there) cuda 12.1. It shouldn't be looking for cuda 11 if you really freshly installed with CUDA_HOME and that PIP stuff set.

A last thing I may have done is to literally relink the cuda directory:

cd /usr/local/
sudo rm -rf /usr/local/cuda # if link only
sudo ln -s /usr/local/cuda-12.1 cuda

But make sure all your cuda stuff is consistent.

Guruprasad68 · 2024-02-12T19:15:20Z

#1101 (comment)
It will be great if the installation steps can be updated with this.

dancasas · 2024-02-15T16:32:05Z

Seems pretty random, but I got rid of this error by just commenting out the only line in LLaVA/llava/__init__.py

#from .model import LlavaLlamaForCausalLM

Now my installation is running correctly. Can anybody explain?

lukaemon · 2024-02-17T19:30:36Z

Same problem here.

Thanks, deepspeed==0.12.6 works for me in my local setting, so basically I found that if you need to train the model, running the below command is sufficient?
pip install -e ".[train]"
pip install flash-attn --no-build-isolation --no-cache-dir

zengxingchen · 2024-02-18T06:19:26Z

pip uninstall  flash-attn
pip install -e ".[train]"
pip install flash-attn --no-build-isolation --no-cache-dir

The above order worked for me. The order is important.

sucongCJS · 2024-02-19T07:42:08Z

Seems pretty random, but I got rid of this error by just commenting out the only line in LLaVA/llava/__init__.py
#from .model import LlavaLlamaForCausalLM
Now my installation is running correctly. Can anybody explain?

I tried but,
NameError: name 'LlavaLlamaForCausalLM' is not defined

nrikoh · 2024-02-28T12:59:24Z

In my and the above case the deepspeed error was hit with inference too.

I didn't check exactly which deepspeed version was required. I just did pip install accelerate deepspeed --upgrade and it started working, so I wanted to remember the versions that worked so wrote those down.

this works for me ,thanks!

20191864218 · 2024-03-01T05:06:39Z

I am also having the same error. I have found that this error occurs depending on the order in which the packages are installed, but have not been able to determine the cause. If I create a new environment and then install only LLaVA, it works correctly, but I would like to use it in combination with other modules, so I would like to see this bug fixed.

I'm also encountering this issue. Have you resolved it?

frankRenlf · 2024-03-05T10:58:44Z

'''
pip uninstall flash-attn
pip install -e ".[train]"
pip install flash-attn --no-build-isolation --no-cache-dir
'''
this works, many thanks

aymenabid-lab · 2024-03-06T07:30:02Z

fro me it don't work!
(llava) C:\Users\aymen\LLaVA>pip uninstall flash-attn
WARNING: Skipping flash-attn as it is not installed.

(llava) C:\Users\aymen\LLaVA>pip install -e ".[train]"
gives:
Collecting deepspeed==0.12.6 (from llava==1.2.2.post1)
Downloading deepspeed-0.12.6.tar.gz (1.2 MB)
---------------------------------------- 1.2/1.2 MB 1.4 MB/s eta 0:00:00
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [15 lines of output]
test.c
LINK : fatal error LNK1181: cannot open input file 'aio.lib'
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "C:\Users\aymen\AppData\Local\Temp\pip-install-554dhcbc\deepspeed_766af2b6636b428d9bb97f1b3acb1da1\setup.py", line 182, in
abort(f"Unable to pre-compile {op_name}")
File "C:\Users\aymen\AppData\Local\Temp\pip-install-554dhcbc\deepspeed_766af2b6636b428d9bb97f1b3acb1da1\setup.py", line 52, in abort
assert False, msg
AssertionError: Unable to pre-compile async_io
DS_BUILD_OPS=1
[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] One can disable async_io with DS_BUILD_AIO=0
[ERROR] Unable to pre-compile async_io
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

And
pip install flash-attn --no-build-isolation --no-cache-dir
gives
Building wheels for collected packages: flash-attn
Building wheel for flash-attn (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [215 lines of output]
fatal: not a git repository (or any of the parent directories): .git

  torch.__version__  = 2.1.2+cu121


  C:\ProgramData\anaconda3\envs\llava\lib\site-packages\setuptools\__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
  !!

          ********************************************************************************
          Requirements should be satisfied by a PEP 517 installer.
          If you are using pip, you can try `pip install --use-pep517`.
          ********************************************************************************

  !!
    dist.fetch_build_eggs(dist.setup_requires)
  running bdist_wheel
  Guessing wheel URL:  https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.6/flash_attn-2.5.6+cu122torch2.1cxx11abiFALSE-cp310-cp310-win_amd64.whl
  Precompiled wheel not found. Building from source...
  C:\ProgramData\anaconda3\envs\llava\lib\site-packages\torch\utils\cpp_extension.py:502: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))

patrick-tssn · 2024-03-13T13:17:54Z

I got rid of the try-except block in llava/model/__init__.py and then reinstalled from root with pip install -e . and the error went away for me
before:

try:
    from .language_model.llava_llama import LlavaLlamaForCausalLM, LlavaConfig
    from .language_model.llava_mpt import LlavaMptForCausalLM, LlavaMptConfig
    from .language_model.llava_mistral import LlavaMistralForCausalLM, LlavaMistralConfig
except:
    pass

after:

from .language_model.llava_llama import LlavaLlamaForCausalLM, LlavaConfig
from .language_model.llava_mpt import LlavaMptForCausalLM, LlavaMptConfig
from .language_model.llava_mistral import LlavaMistralForCausalLM, LlavaMistralConfig

Tried and now i get this error:

python3 -m llava.serve.controller --host 0.0.0.0 --port 10000
[2024-02-08 14:52:59,501] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)
/home/tobias/.local/lib/python3.10/site-packages/pydantic/_internal/_config.py:322: UserWarning: Valid config keys have changed in V2:
* 'allow_population_by_field_name' has been renamed to 'populate_by_name'
* 'validate_all' has been renamed to 'validate_default'
  warnings.warn(message, UserWarning)
/home/tobias/.local/lib/python3.10/site-packages/pydantic/_internal/_fields.py:151: UserWarning: Field "model_persistence_threshold" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
Traceback (most recent call last):
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1364, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/generation/utils.py", line 28, in <module>
    from ..integrations.deepspeed import is_deepspeed_zero3_enabled
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/integrations/deepspeed.py", line 49, in <module>
    from accelerate.utils.deepspeed import HfDeepSpeedConfig as DeepSpeedConfig
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/__init__.py", line 3, in <module>
    from .accelerator import Accelerator
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/accelerator.py", line 35, in <module>
    from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/checkpointing.py", line 24, in <module>
    from .utils import (
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/utils/__init__.py", line 133, in <module>
    from .launch import (
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/utils/launch.py", line 33, in <module>
    from ..utils.other import is_port_in_use, merge_dicts
  File "/home/tobias/.local/lib/python3.10/site-packages/accelerate/utils/other.py", line 30, in <module>
    from deepspeed import DeepSpeedEngine
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/__init__.py", line 16, in <module>
    from . import module_inject
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/module_inject/__init__.py", line 6, in <module>
    from .replace_module import replace_transformer_layer, revert_transformer_layer, ReplaceWithTensorSlicing, GroupQuantizer, generic_injection
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py", line 792, in <module>
    from ..pipe import PipelineModule
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/pipe/__init__.py", line 6, in <module>
    from ..runtime.pipe import PipelineModule, LayerSpec, TiedLayerSpec
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/pipe/__init__.py", line 6, in <module>
    from .module import PipelineModule, LayerSpec, TiedLayerSpec
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/pipe/module.py", line 19, in <module>
    from ..activation_checkpointing import checkpointing
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/activation_checkpointing/checkpointing.py", line 25, in <module>
    from deepspeed.runtime.config import DeepSpeedConfig
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/config.py", line 29, in <module>
    from .zero.config import get_zero_config, ZeroStageEnum
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/zero/__init__.py", line 6, in <module>
    from .partition_parameters import ZeroParamType
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/zero/partition_parameters.py", line 616, in <module>
    class Init(InsertPostInitMethodToModuleSubClasses):
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/zero/partition_parameters.py", line 618, in Init
    param_persistence_threshold = get_config_default(DeepSpeedZeroConfig, "param_persistence_threshold")
  File "/home/tobias/.local/lib/python3.10/site-packages/deepspeed/runtime/config_utils.py", line 116, in get_config_default
    field_name).required, f"'{field_name}' is a required field and does not have a default value"
AttributeError: 'FieldInfo' object has no attribute 'required'. Did you mean: 'is_required'?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1364, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 40, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 44, in <module>
    from .generation import GenerationConfig, GenerationMixin
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1354, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1366, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
'FieldInfo' object has no attribute 'required'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.10/runpy.py", line 110, in _get_module_details
    __import__(pkg_name)
  File "/mnt/a/KI/LLaVA/llava/__init__.py", line 1, in <module>
    from .model import LlavaLlamaForCausalLM
  File "/mnt/a/KI/LLaVA/llava/model/__init__.py", line 2, in <module>
    from .language_model.llava_llama import LlavaLlamaForCausalLM, LlavaConfig
  File "/mnt/a/KI/LLaVA/llava/model/language_model/llava_llama.py", line 21, in <module>
    from transformers import AutoConfig, AutoModelForCausalLM, \
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1355, in __getattr__
    value = getattr(module, name)
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1354, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/home/tobias/.local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1366, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
'FieldInfo' object has no attribute 'required'

I believe that in certain situations, it would be beneficial to disable the try-except mechanism during the debugging process, particularly after making modifications to the original codebase. For instance, in this scenario, the error arises due to the binding of the local package within the Conda environment, so you can see this for help.

jam-cc · 2024-04-10T20:01:53Z

In my and the above case the deepspeed error was hit with inference too.

I didn't check exactly which deepspeed version was required. I just did pip install accelerate deepspeed --upgrade and it started working, so I wanted to remember the versions that worked so wrote those down.

It works for me, thanks!

CHELSEA234 · 2024-04-17T18:53:27Z

A quick update. I followed the installment routine and attended the discussion on this page, but I still ran into the problem.
I noticed one potential reason is the torch version, and then I followed this link to get rid of the problem:
pytorch/pytorch#111469 (comment)

ZhengShuozai · 2024-07-01T03:31:05Z

During the training process, I always encountered this problem. Later, based on my CUDA=11.7, I downgraded the torch version and also downgraded the flash attn version, and the problem was solved.
Currently torch=1.13.1 flash-attn==2.3 tokenizers == 0.11.4

Fixed due to haotian-liu#1101 (comment)

pervaizniazi · 2024-07-26T16:37:06Z

I solved the issue by changing Python version:

replacing
conda create -n llava python=3.10 -y
with
conda create -n llava python=3.9 -y

haotian-liu closed this as completed Feb 28, 2024

zzxslp mentioned this issue Mar 7, 2024

[Question] ImportError: cannot import name 'LlavaLlamaForCausalLM' from 'llava.model' (/root/LLaVA/llava/model/__init__.py) #1208

Open

zhaohm14 mentioned this issue Mar 18, 2024

[Usage] ImportError: cannot import name 'LlavaLlamaForCausalLM' from 'llava.model' #1291

Closed

This was referenced May 22, 2024

cannot import name 'LlavaLlamaForCausalLM' from 'llava.model' LLaVA-VL/LLaVA-NeXT#28

Open

fix: Fix class not found error LLaVA-VL/LLaVA-NeXT#29

Closed

l-show mentioned this issue Jun 13, 2024

[版本冲突]环境配置要求版本冲突 #1558

Open

melihsozdinler added a commit to melihsozdinler/LLaVA that referenced this issue Jul 11, 2024

Update __init__.py

40cd5c8

Fixed due to haotian-liu#1101 (comment)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Usage] ImportError: cannot import name 'LlavaLlamaForCausalLM' from 'llava.model' #1101

[Usage] ImportError: cannot import name 'LlavaLlamaForCausalLM' from 'llava.model' #1101

TobiasJu commented Feb 8, 2024 •

edited

Loading

anubhavashok commented Feb 8, 2024 •

edited

Loading

pseudotensor commented Feb 8, 2024 •

edited

Loading

celll1 commented Feb 8, 2024

TobiasJu commented Feb 8, 2024

pseudotensor commented Feb 8, 2024

haotian-liu commented Feb 8, 2024 •

edited

Loading

pseudotensor commented Feb 8, 2024

haotian-liu commented Feb 8, 2024

pseudotensor commented Feb 8, 2024

haotian-liu commented Feb 8, 2024

nitzanguetta commented Feb 8, 2024

pseudotensor commented Feb 9, 2024

Guruprasad68 commented Feb 12, 2024

dancasas commented Feb 15, 2024

lukaemon commented Feb 17, 2024

zengxingchen commented Feb 18, 2024

sucongCJS commented Feb 19, 2024

nrikoh commented Feb 28, 2024

20191864218 commented Mar 1, 2024

frankRenlf commented Mar 5, 2024

aymenabid-lab commented Mar 6, 2024 •

edited

Loading

patrick-tssn commented Mar 13, 2024 •

edited

Loading

jam-cc commented Apr 10, 2024

CHELSEA234 commented Apr 17, 2024

ZhengShuozai commented Jul 1, 2024

pervaizniazi commented Jul 26, 2024

[Usage] ImportError: cannot import name 'LlavaLlamaForCausalLM' from 'llava.model' #1101

[Usage] ImportError: cannot import name 'LlavaLlamaForCausalLM' from 'llava.model' #1101

Comments

TobiasJu commented Feb 8, 2024 • edited Loading

Describe the issue

anubhavashok commented Feb 8, 2024 • edited Loading

pseudotensor commented Feb 8, 2024 • edited Loading

celll1 commented Feb 8, 2024

TobiasJu commented Feb 8, 2024

pseudotensor commented Feb 8, 2024

haotian-liu commented Feb 8, 2024 • edited Loading

pseudotensor commented Feb 8, 2024

haotian-liu commented Feb 8, 2024

pseudotensor commented Feb 8, 2024

haotian-liu commented Feb 8, 2024

nitzanguetta commented Feb 8, 2024

pseudotensor commented Feb 9, 2024

Guruprasad68 commented Feb 12, 2024

dancasas commented Feb 15, 2024

lukaemon commented Feb 17, 2024

zengxingchen commented Feb 18, 2024

sucongCJS commented Feb 19, 2024

nrikoh commented Feb 28, 2024

20191864218 commented Mar 1, 2024

frankRenlf commented Mar 5, 2024

aymenabid-lab commented Mar 6, 2024 • edited Loading

patrick-tssn commented Mar 13, 2024 • edited Loading

jam-cc commented Apr 10, 2024

CHELSEA234 commented Apr 17, 2024

ZhengShuozai commented Jul 1, 2024

pervaizniazi commented Jul 26, 2024

TobiasJu commented Feb 8, 2024 •

edited

Loading

anubhavashok commented Feb 8, 2024 •

edited

Loading

pseudotensor commented Feb 8, 2024 •

edited

Loading

haotian-liu commented Feb 8, 2024 •

edited

Loading

aymenabid-lab commented Mar 6, 2024 •

edited

Loading

patrick-tssn commented Mar 13, 2024 •

edited

Loading