ModuleNotFoundError: No module named 'packaging' ERROR

[IdoAmit198]

Describe the issue

Hi everyone, I'm trying to clone onnxruntime repo and install (in order to later use the run_benchmark.sh script), but facing a really weird issue when I try to install. It errors on a missing package which I do have installed.

I already tried the solutions described here without a success.

Error and Output:

Processing /store/code/gpt2/quant/onnxruntime
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
          main()
        File "/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/venv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-31p956_z/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-31p956_z/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-31p956_z/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 484, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-31p956_z/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 17, in <module>
      ModuleNotFoundError: No module named 'packaging'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Please read To reproduce section to see what I did to have that error.

Expected behavior:

installation complete successfully.

I also attach my installed packages list:

installed_packages.txt

Would really appreciate any help on the topic!

To reproduce

I'm cloning the repo:

git clone https://github.com/microsoft/onnxruntime.git

I try to install:

pip install .

Then errors

And that's it, here I get the error you can see above in my description..

Urgency

No response

Platform

Linux

OS Version

20.04.4 LTS (Focal Fossa)

ONNX Runtime Installation

Other / Unknown

ONNX Runtime Version or Commit ID

onnxruntime Tag=1.13.1 , Commit ID=#13448

ONNX Runtime API

Python

Architecture

X86

Execution Provider

Other / Unknown

Execution Provider Library Version

No response

[wangyems]

please check if "packaging" exists in your python's site-packages

[IdoAmit198]

Hi @wangyems , Thanks for the response.
I checked and it exist in my python's site-packages.

[hungrymonkey]

pip --version
pip 23.0.1 from /home/doof/gitrepo/dladata/venv/lib64/python3.10/site-packages/pip (python 3.10)
python3 --version
Python 3.10.10

pip install packaging
Requirement already satisfied: packaging in /home/doof/gitrepo/dladata/venv/lib/python3.10/site-packages (23.0)

 pip install git+https://github.com/microsoft/onnxruntime.git


 lsb_release  -a
LSB Version:	n/a
Distributor ID:	openSUSE
Description:	openSUSE Tumbleweed
Release:	20230313
Codename:	n/a

Running command git clone --filter=blob:none --quiet https://github.com/microsoft/onnxruntime.git /tmp/pip-req-build-xy9dn4_d
  Resolved https://github.com/microsoft/onnxruntime.git to commit 5ac3a37be5d5bd25d167c2127b9c85f4b53668e7
  Running command git submodule update --init --recursive -q
j  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/home/doof/gitrepo/dladata/venv/lib64/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/doof/gitrepo/dladata/venv/lib64/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/doof/gitrepo/dladata/venv/lib64/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-w28k5dyg/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-w28k5dyg/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-w28k5dyg/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 484, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-w28k5dyg/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 17, in <module>
      ModuleNotFoundError: No module named 'packaging'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

pip list
Package                  Version
------------------------ -----------
aiohttp                  3.8.4
aiosignal                1.3.1
altair                   4.2.2
annoy                    1.17.1
anyio                    3.6.2
async-timeout            4.0.2
attrs                    22.2.0
beautifulsoup4           4.11.2
blinker                  1.5
cachetools               5.3.0
certifi                  2022.12.7
cffi                     1.15.1
charset-normalizer       3.1.0
click                    8.1.3
cmake                    3.26.0
coloredlogs              15.0.1
cryptography             39.0.2
datasets                 2.10.1
decorator                5.1.1
dill                     0.3.6
elastic-transport        8.4.0
elasticsearch            8.6.2
entrypoints              0.4
faiss-cpu                1.7.3
fastapi                  0.94.1
fasteners                0.18
fasttext                 0.9.2
filelock                 3.10.0
flatbuffers              23.3.3
fonttools                4.39.2
frozenlist               1.3.3
fsspec                   2023.3.0
gitdb                    4.0.10
GitPython                3.1.31
h11                      0.14.0
hnswlib                  0.7.0
huggingface-hub          0.13.2
humanfriendly            10.0
idna                     3.4
importlib-metadata       6.0.0
Jinja2                   3.1.2
joblib                   1.2.0
jsonschema               4.17.3
lit                      15.0.7
lxml                     4.9.2
lz4                      4.3.2
markdown-it-py           2.2.0
MarkupSafe               2.1.2
mdurl                    0.1.2
mpmath                   1.3.0
multidict                6.0.4
multiprocess             0.70.14
networkx                 3.0
nltk                     3.8.1
numpy                    1.24.2
nvidia-cublas-cu11       11.10.3.66
nvidia-cuda-cupti-cu11   11.7.101
nvidia-cuda-nvrtc-cu11   11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11        8.5.0.96
nvidia-cufft-cu11        10.9.0.58
nvidia-curand-cu11       10.2.10.91
nvidia-cusolver-cu11     11.4.0.1
nvidia-cusparse-cu11     11.7.4.91
nvidia-nccl-cu11         2.14.3
nvidia-nvtx-cu11         11.7.91
onnxruntime              1.14.1
packaging                23.0
pandas                   1.5.3
paperai                  2.2.0
paperetl                 2.2.0
pdf-annotate             0.12.0
pdfminer.six             20221105
pdfrw                    0.4
Pillow                   9.4.0
pip                      23.0.1
protobuf                 3.20.3
pyarrow                  11.0.0
pybind11                 2.10.4
pycparser                2.21
pydantic                 1.10.6
pydeck                   0.8.0
Pygments                 2.14.0
pymagnitude-lite         0.1.143
Pympler                  1.0.1
pyrsistent               0.19.3
python-dateutil          2.8.2
pytz                     2022.7.1
pytz-deprecation-shim    0.1.0.post0
PyYAML                   6.0
regex                    2022.10.31
requests                 2.28.2
responses                0.18.0
rich                     13.3.2
rocketchat-async         1.0.1
scikit-learn             1.2.2
scipy                    1.10.1
semver                   2.13.0
sentence-transformers    2.2.2
sentencepiece            0.1.97
setuptools               65.5.0
six                      1.16.0
smmap                    5.0.0
sniffio                  1.3.0
soupsieve                2.4
starlette                0.26.1
streamlit                1.20.0
streamlit-chat           0.0.2.2
sympy                    1.11.1
text2digits              0.1.0
threadpoolctl            3.1.0
tokenizers               0.13.2
toml                     0.10.2
toolz                    0.12.0
torch                    2.0.0
torchvision              0.15.1
tornado                  6.2
tqdm                     4.65.0
transformers             4.27.1
triton                   2.0.0
txtai                    5.5.0
txtchat                  0.2.0
txtmarker                1.0.0
typing_extensions        4.5.0
tzdata                   2022.7
tzlocal                  4.2
urllib3                  1.26.15
uvicorn                  0.21.1
validators               0.20.0
watchdog                 2.3.1
websockets               10.4
wheel                    0.40.0
xxhash                   3.2.0
yarl                     1.8.2
zipp                     3.15.0

Same error here.

[belm0]

I think this is a regression of onnxruntime 1.13

The setup.py now depends on 3rd party package packaging, without declaring the dependency. I believe it should be specified in pyproject.toml:

[build-system]
requires = ["packaging"]

[bit-scientist]

@belm0 putting packaging into pyproject.toml didn't help. I also tried installing the package with conda and apt-get. Neither worked. I wonder how one can handle this.

[hieunct]

Have

@belm0 putting packaging into pyproject.toml didn't help. I also tried installing the package with conda and apt-get. Neither worked. I wonder how one can handle this.

Have you found the solution for this? I have run into the same issue with Python3.10. Thanks!

[saman-rahbar]

Any update on this? I'm also getting the same error using python3.8. None of the other work around solutions helped. (E.g. including the setup file, pip installing the packaging library, etc...)

[justinchuby]

pip install . is not supported I don't think. I suggest following https://onnxruntime.ai/docs/build/inferencing.html#common-build-instructions to build and install the python wheel.

[sinapy]

I have the same issue when running pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" --global-option="--deprecated_fused_adam" --global-option="--xentropy" --global-option="--fast_multihead_attn" ./

from fairseq https://github.com/facebookresearch/fairseq#requirements-and-installation

error:

  Running command Getting requirements to build wheel
  Traceback (most recent call last):
    File "C:\Users\301212298\source\repos\unilm\venv\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 363, in <module>
      main()
    File "C:\Users\301212298\source\repos\unilm\venv\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:\Users\301212298\source\repos\unilm\venv\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 130, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\301212~1\AppData\Local\Temp\pip-build-env-rckcnzxb\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "C:\Users\301212~1\AppData\Local\Temp\pip-build-env-rckcnzxb\overlay\Lib\site-packages\setuptools\build_meta.py", line 323, in _get_build_requires
      self.run_setup()
    File "C:\Users\301212~1\AppData\Local\Temp\pip-build-env-rckcnzxb\overlay\Lib\site-packages\setuptools\build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 5, in <module>
  ModuleNotFoundError: No module named 'packaging'
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'C:\Users\301212298\source\repos\unilm\venv\Scripts\python.exe' 'C:\Users\301212298\source\repos\unilm\venv\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' get_requires_for_build_wheel 'C:\Users\301212~1\AppData\Local\Temp\tmpxjtt5ljt'
  cwd: C:\Users\301212298\source\repos\unilm\trocr\apex
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error

[erh94]

Any update on this, I also getting the issue with master

commit 9da3e36

Steps followed :

git clone https://github.com/microsoft/onnxruntime.git
cd onnxruntime
pip install -e .

Python 3.8.10

Error :

Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... error
  error: subprocess-exited-with-error

  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      Traceback (most recent call last):
        File "/local/mnt/workspace/training/pipeline/env/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/local/mnt/workspace/training/pipeline/env/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/local/mnt/workspace/training/pipeline/env/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 132, in get_requires_for_build_editable
          return hook(config_settings)
        File "/tmp/pip-build-env-vkbqyp28/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 441, in get_requires_for_build_editable
          return self.get_requires_for_build_wheel(config_settings)
        File "/tmp/pip-build-env-vkbqyp28/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-vkbqyp28/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-vkbqyp28/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 480, in run_setup
          super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-vkbqyp28/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 18, in <module>
      ModuleNotFoundError: No module named 'packaging'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

> pip show packaging
Name: packaging
Version: 23.2
Summary: Core utilities for Python packages
Home-page:
Author:
Author-email: Donald Stufft <donald@stufft.io>
License:
Location: /local/mnt/workspace/training/pipeline/env/lib/python3.8/site-packages
Requires:
Required-by: accelerate, datasets, docker, evaluate, gunicorn, huggingface-hub, ipykernel, jupyter_server, jupyterlab, jupyterlab_server, matplotlib, mlflow, nbconvert, onnxruntime-training, peft, qtconsole, QtPy, transformers

Thanks

[dhivyeshrk]

Any update on this?

      Traceback (most recent call last):
        File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-20pdylp3/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-20pdylp3/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-20pdylp3/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 5, in <module>
      ModuleNotFoundError: No module named 'packaging'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

I have verified the installation of a non-conflicting version of 'packaging'. Thanks.

[leeedwina430]

any update on this issue? I encountered the same one here

[justinchuby]

pip install . is not supported I don't think. I suggest following onnxruntime.ai/docs/build/inferencing.html#common-build-instructions to build and install the python wheel.

Please see the build instructions.

Long term, it would be really nice if we can support building directly from pip install . @snnn

[tagMatrix4]

use:
pip install -v --no-build-isolation -- no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext"

[xty435768]

use: pip install -v --no-build-isolation -- no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext"

thanks! this solution works for me

[roshanjoebenny]

pip install -v --no-build-isolation -- no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" -> When I am executing this line I am getting an error thats given below

ERROR: Invalid requirement: '--global-option=--cpp_ext'
Hint: = is not a valid operator. Did you mean == ?

[angeloskanatas]

pip install -v --no-build-isolation -- no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" -> When I am executing this line I am getting an error thats given below

ERROR: Invalid requirement: '--global-option=--cpp_ext' Hint: = is not a valid operator. Did you mean == ?

do: pip install -v --no-build-isolation --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

[wanan0414]

(swintssl) C:\Users\26804\apex>pip install -v --no-build-isolation --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
Using pip 23.3.1 from E:\Anaconda\envs\swintssl\lib\site-packages\pip (python 3.8)
DEPRECATION: --build-option and --global-option are deprecated. pip 24.0 will enforce this behaviour change. A possible replacement is to use --config-settings. Discussion can be found at pypa/pip#11859
WARNING: Implying --no-binary=:all: due to the presence of --build-option / --global-option.
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Processing c:\users\26804\apex
Running command python setup.py egg_info
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "C:\Users\26804\apex\setup.py", line 132, in
_, bare_metal_version = get_cuda_bare_metal_version(CUDA_HOME)
File "C:\Users\26804\apex\setup.py", line 19, in get_cuda_bare_metal_version
raw_output = subprocess.check_output([cuda_dir + "/bin/nvcc", "-V"], universal_newlines=True)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

torch.version = 1.7.1

error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
full command: 'E:\Anaconda\envs\swintssl\python.exe' -c '
exec(compile('"'"''"'"''"'"'

This is -- a caller that pip uses to run setup.py

- It imports setuptools before invoking setup.py, to enable projects that directly

import from distutils.core to work with newer packaging standards.

- It provides a clear error message when setuptools is not installed.

- It sets sys.argv[0] to the underlying setup.py, when invoking setup.py so

setuptools doesn'"'"'t think the script is -c. This avoids the following warning:

manifest_maker: standard file '"'"'-c'"'"' not found".

- It generates a shim setup.py, for handling setup.cfg-only projects.

import os, sys, tokenize

try:
import setuptools
except ImportError as error:
print(
"ERROR: Can not execute setup.py since setuptools is not available in "
"the build environment.",
file=sys.stderr,
)
sys.exit(1)

file = %r
sys.argv[0] = file

if os.path.exists(file):
filename = file
with tokenize.open(file) as f:
setup_py_code = f.read()
else:
filename = ""
setup_py_code = "from setuptools import setup; setup()"

exec(compile(setup_py_code, filename, "exec"))
'"'"''"'"''"'"' % ('"'"'C:\Users\26804\apex\setup.py'"'"',), "", "exec"))' egg_info --egg-base 'C:\Users\26804\AppData\Local\Temp\pip-pip-egg-info-yxlkmtz1'
cwd: C:\Users\26804\apex
Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

pip install -v --no-build-isolation -- no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" -> When I am executing this line I am getting an error thats given below
ERROR: Invalid requirement: '--global-option=--cpp_ext' Hint: = is not a valid operator. Did you mean == ?

do: pip install -v --no-build-isolation --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

[roshanjoebenny]

pip install -v --no-build-isolation --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

I have executed above given command still I am getting the error

TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Preparing metadata (pyproject.toml) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output

Can anyone please help to solve this issue