ImportError: cannot import name 'neuron_xla_compile' from 'libneuronxla' (/home/mlvenv/lib64/python3.8/site-packages/libneuronxla/__init__.py)

[Bhuvanesh09]

Current PIP installations:

absl-py==1.4.0
accelerate==0.22.0
aio-multiprocess==1.3.4
aiobotocore==2.1.1
aiofiles==0.8.0
aiohttp==3.7.0
aioitertools==0.11.0
aiokafka==0.8.0
aiologger==0.6.1
anyio==3.6.2
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.2.3
asgard==5.0.0
asttokens==2.4.0
async-generator==1.10
async-lru==2.0.4
async-timeout==3.0.1
attrs==22.2.0
aws-neuronx-runtime-discovery==2.9
awscli==1.29.39
Babel==2.12.1
backcall==0.2.0
backports.zoneinfo==0.2.1
beautifulsoup4==4.6.0
bert-serving-client==1.8.3
bert-serving-server==1.8.3
bleach==6.0.0
boto3==1.20.24
botocore==1.31.39
cachetools==4.0.0
certifi==2022.12.7
cffi==1.15.1
chardet==3.0.4
charset-normalizer==2.1.1
click==8.1.3
cloud-tpu-client==0.10
colorama==0.4.4
coloredlogs==15.0.1
comm==0.1.4
datasets==2.14.4
dateparser==1.1.1
debugpy==1.6.7.post1
decorator==5.1.1
defusedxml==0.7.1
dill==0.3.7
docopt==0.6.2
docutils==0.16
easydict==1.9
emoji==0.6.0
exceptiongroup==1.1.3
executing==1.2.0
fastjsonschema==2.18.0
filelock==3.10.0
fqdn==1.5.1
fsspec==2023.9.0
ftfy==6.1.1
gcsfs==2021.10.1
google-api-core==1.34.0
google-api-python-client==1.8.0
google-auth==2.16.2
google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0
google-cloud-core==2.3.2
google-cloud-storage==2.7.0
google-crc32c==1.5.0
google-resumable-media==2.4.1
googleapis-common-protos==1.56.2
GPUtil==1.4.0
grpcio==1.48.2
grpcio-reflection==1.37.0
grpcio-tools==1.37.0
h11==0.12.0
hbmqtt==0.9.6
htmllaundry==2.2
httpcore==0.14.7
httplib2==0.22.0
httpx==0.22.0
huggingface-hub==0.16.4
humanfriendly==10.0
idna==3.4
importlib-metadata==6.8.0
importlib-resources==6.0.1
iniconfig==2.0.0
intuition-api==5.0.0
ipykernel==6.25.2
ipython==8.12.2
ipython-genutils==0.2.0
ipywidgets==8.1.0
islpy==2022.1.1
isoduration==20.11.0
jedi==0.19.0
Jinja2==3.1.2
jmespath==0.10.0
joblib==1.2.0
json5==0.9.14
jsonpointer==2.4
jsonschema==4.19.0
jsonschema-specifications==2023.7.1
jupyter==1.0.0
jupyter-console==6.6.3
jupyter-events==0.7.0
jupyter-lsp==2.2.0
jupyter_client==8.3.1
jupyter_core==5.3.1
jupyter_server==2.7.3
jupyter_server_terminals==0.4.4
jupyterlab==4.0.5
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.8
jupyterlab_server==2.24.0
kafka-python==2.0.2
langcodes==3.3.0
libneuronxla==0.5.207
lockfile==0.12.2
lxml==4.9.2
MarkupSafe==2.1.3
matplotlib-inline==0.1.6
mistune==3.0.1
model-store==5.0.0
mongoengine==0.24.0
mongomock==3.20.0
mpmath==1.3.0
msgpack==1.0.5
multi-key-dict==2.0.3
multidict==6.0.4
multiprocess==0.70.15
nbclient==0.8.0
nbconvert==7.8.0
nbformat==5.9.2
nest-asyncio==1.5.7
networkx==2.6.3
neuronx-cc==2.6.0.19+3d819e565
neuronx-distributed==0.1
neuronx-hwm==2.6.0.0+826e77395
nltk==3.7
notebook==7.0.3
notebook_shim==0.2.3
numpy==1.21.6
nvidia-cublas-cu11==11.10.3.66
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-runtime-cu11==11.7.99
nvidia-cudnn-cu11==8.5.0.96
oauth2client==4.1.3
oauthlib==3.2.2
optimum==1.12.0
optimum-neuron==0.0.10
overrides==1.9
packaging==23.0
pandas==2.0.3
pandocfilters==1.5.0
parso==0.8.3
passlib==1.7.4
pbr==5.11.1
pexpect==4.8.0
pgzip==0.3.5
pickleshare==0.7.5
Pillow==9.0.1
pipelining==5.0.0
pkgutil_resolve_name==1.3.10
platformdirs==3.10.0
pluggy==1.3.0
prometheus-client==0.17.1
prompt-toolkit==3.0.39
protobuf==3.19.6
psutil==5.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
pyarrow==13.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
pydot==1.4.1
pyenchant==3.1.1
Pygments==2.16.1
pymongo==4.0
pymsteams==0.2.2
pyparsing==3.0.9
pytest==7.4.0
python-daemon==3.0.1
python-dateutil==2.8.2
python-jenkins==1.5.0
python-json-logger==2.0.7
pytz==2022.7.1
pytz-deprecation-shim==0.1.0.post0
PyYAML==5.3.1
pyzmq==25.0.2
qtconsole==5.4.4
QtPy==2.4.0
referencing==0.30.2
regex==2022.3.2
requests==2.28.1
requests-oauthlib==1.3.1
requests-unixsocket==0.3.0
rfc3339-validator==0.1.4
rfc3986==1.5.0
rfc3986-validator==0.1.1
rpds-py==0.10.2
rsa==4.7.2
Rx==3.0.1
s3transfer==0.6.2
safetensors==0.3.3
schedule==1.1.0
scikit-learn==0.23.1
scipy==1.7.3
Send2Trash==1.8.2
sentencepiece==0.1.99
sentinels==1.0.0
six==1.16.0
slack-sdk==3.15.1
sniffio==1.3.0
stack-data==0.6.2
sympy==1.12
termcolor==2.2.0
terminado==0.17.1
threadpoolctl==3.1.0
tinycss2==1.2.1
tokenizers==0.12.1
tomli==2.0.1
tools-commons==5.0.0
tools-config==5.0.0
tools-frameworks==5.0.0
tools-fs==5.0.0
tools-messaging==5.0.0
tools-nlp==5.0.0
torch==1.13.1
torch-neuronx==1.13.1.1.7.0
torch-xla==1.13.1+torchneuron6
torchvision==0.14.1
tornado==6.2
tqdm==4.65.0
traitlets==5.9.0
transformers==4.33.0
transformers-neuronx==0.6.106
transitions==0.9.0
ttldict==0.3.0
typing_extensions==4.5.0
tzdata==2022.7
tzlocal==4.3
Unidecode==1.0.23
uri-template==1.3.0
uritemplate==3.0.1
urllib3==1.26.15
wcwidth==0.2.6
webcolors==1.13
webencodings==0.5.1
websocket-client==0.56.0
websockets==8.1
wget==3.2
widgetsnbextension==4.0.8
wordfreq==2.5.1
wrapt==1.15.0
xlrd==1.2.0
xxhash==3.3.0
yarl==1.8.2
zipp==3.16.2

Error on command:

model = NeuronModelForCausalLM.from_pretrained("gpt2", export=True)

Stack Trace:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[2], line 1
----> 1 model = NeuronModelForCausalLM.from_pretrained("gpt2", export=True)

File /home/mluser/mlvenv/lib64/python3.8/site-packages/optimum/modeling_base.py:372, in OptimizedModel.from_pretrained(cls, model_id, export, force_download, use_auth_token, cache_dir, subfolder, config, local_files_only, trust_remote_code, revision, **kwargs)
    369     trust_remote_code = False
    371 from_pretrained_method = cls._from_transformers if export else cls._from_pretrained
--> 372 return from_pretrained_method(
    373     model_id=model_id,
    374     config=config,
    375     revision=revision,
    376     cache_dir=cache_dir,
    377     force_download=force_download,
    378     use_auth_token=use_auth_token,
    379     subfolder=subfolder,
    380     local_files_only=local_files_only,
    381     trust_remote_code=trust_remote_code,
    382     **kwargs,
    383 )

File /home/mluser/mlvenv/lib64/python3.8/site-packages/optimum/neuron/modeling_decoder.py:110, in NeuronDecoderModel._from_transformers(cls, model_id, config, use_auth_token, revision, force_download, cache_dir, subfolder, local_files_only, trust_remote_code, task, batch_size, num_cores, auto_cast_type, **kwargs)
    107     task = TasksManager.infer_task_from_model(cls.auto_model_class)
    109 # Instantiate the exporter for the specified configuration and task
--> 110 exporter = get_exporter(config, task)
    112 # Split kwargs between model and neuron args
    113 model_kwargs, neuron_kwargs = exporter.split_kwargs(**kwargs)

File /home/mluser/mlvenv/lib64/python3.8/site-packages/optimum/neuron/modeling_decoder.py:49, in get_exporter(config, task)
     48 def get_exporter(config, task):
---> 49     return TasksManager.get_exporter_config_constructor(model_type=config.model_type, exporter="neuron", task=task)()

File /home/mluser/mlvenv/lib64/python3.8/site-packages/optimum/exporters/neuron/base.py:346, in NeuronDecoderConfig.__init__(self, task)
    342     raise ModuleNotFoundError(
    343         "The mandatory transformers-neuronx package is missing. Please install optimum[neuronx]."
    344     )
    345 module_name, class_name = self.NEURONX_CLASS.rsplit(".", maxsplit=1)
--> 346 module = importlib.import_module(f"transformers_neuronx.{module_name}")
    347 self._neuronx_class = getattr(module, class_name, None)
    348 if self._neuronx_class is None:

File /usr/lib64/python3.8/importlib/__init__.py:127, in import_module(name, package)
    125             break
    126         level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1014, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:991, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:975, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:671, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:843, in exec_module(self, module)

File <frozen importlib._bootstrap>:219, in _call_with_frames_removed(f, *args, **kwds)

File /home/mluser/mlvenv/lib64/python3.8/site-packages/transformers_neuronx/gpt2/model.py:20
     18 from transformers import PreTrainedModel
     19 from transformers.utils import ModelOutput
---> 20 from transformers_neuronx import decoder
     21 from transformers_neuronx import dtypes
     22 from transformers_neuronx import hlo

File /home/mluser/mlvenv/lib64/python3.8/site-packages/transformers_neuronx/decoder.py:18
     16 import os
     17 import torch
---> 18 from transformers_neuronx import compiler
     19 from transformers_neuronx import dtypes
     20 from transformers_neuronx import hlo

File /home/mluser/mlvenv/lib64/python3.8/site-packages/transformers_neuronx/compiler.py:31
     29 from transformers_neuronx import ops
     30 from transformers_neuronx import parallel
---> 31 from libneuronxla import neuron_xla_compile
     32 from neuronxcc import __version__ as compiler_version
     34 def get_hash_module(hlo_module, flags):
     35     # Hashing is pretty fast and neglegible compared to compilation time

ImportError: cannot import name 'neuron_xla_compile' from 'libneuronxla' (/home/mluser/mlvenv/lib64/python3.8/site-packages/libneuronxla/__init__.py)

[michaelbenayoun]

Did you install the Neuron drivers and tools as documented here.

cc @JingyaHuang @dacorvo

[Bhuvanesh09]

Did you install the Neuron drivers and tools as documented here.

cc @JingyaHuang @dacorvo

Yes, one difference being that I've used an image based out of rockylinux and its run on a pod with Inf2.

[dacorvo]

It looks like you are not running the latest version of torch-neuronx. See a similar issue.

[HuggingFaceDocBuilderDev]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

[HuggingFaceDocBuilderDev]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!