[python] pip install fails starting with tensorrt 8.6.1 release

[ddelange]

Description

Hi 👋

pip install is broken for latest tensorrt:

• tensorrt 8.6.0 posted only wheels to PyPI
• tensorrt 8.6.1 posts only a source distribution to PyPI
• the install of tensorrt 8.6.1 tries to fetch tensorrt_libs==8.6.1 from https://pypi.nvidia.com
• from the traceback below, the latter index seems to be private / not publicly accessible

Environment

irrelevant

Relevant Files

irrelevant

Steps To Reproduce

Simply run a pip install in any venv:

Commands or scripts:

$ pip install tensorrt
Collecting tensorrt
  Downloading tensorrt-8.6.1.tar.gz (16 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: tensorrt
  Building wheel for tensorrt (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [49 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib
      creating build/lib/tensorrt
      copying tensorrt/__init__.py -> build/lib/tensorrt
      running egg_info
      writing tensorrt.egg-info/PKG-INFO
      writing dependency_links to tensorrt.egg-info/dependency_links.txt
      writing requirements to tensorrt.egg-info/requires.txt
      writing top-level names to tensorrt.egg-info/top_level.txt
      reading manifest file 'tensorrt.egg-info/SOURCES.txt'
      adding license file 'LICENSE.txt'
      writing manifest file 'tensorrt.egg-info/SOURCES.txt'
      /Users/ddelange/.pyenv/versions/3.8.10/envs/vv/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      installing to build/bdist.macosx-10.15-x86_64/wheel
      running install
      Looking in indexes: https://pypi.nvidia.com
      ERROR: Could not find a version that satisfies the requirement tensorrt_libs==8.6.1 (from versions: none)
      ERROR: No matching distribution found for tensorrt_libs==8.6.1
      WARNING: There was an error checking the latest version of pip.
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/_b/ldf_nfdx3qndz_5lvxxkchsc0000gn/T/pip-install-8nnh031o/tensorrt_2ea76508e3b64219b3a1a1c3602f7e99/setup.py", line 49, in <module>
          setup(
        File "/Users/ddelange/.pyenv/versions/3.8.10/envs/vv/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/Users/ddelange/.pyenv/versions/3.8.10/lib/python3.8/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/Users/ddelange/.pyenv/versions/3.8.10/lib/python3.8/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/Users/ddelange/.pyenv/versions/3.8.10/lib/python3.8/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/Users/ddelange/.pyenv/versions/3.8.10/envs/vv/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 335, in run
          self.run_command('install')
        File "/Users/ddelange/.pyenv/versions/3.8.10/lib/python3.8/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/Users/ddelange/.pyenv/versions/3.8.10/lib/python3.8/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/private/var/folders/_b/ldf_nfdx3qndz_5lvxxkchsc0000gn/T/pip-install-8nnh031o/tensorrt_2ea76508e3b64219b3a1a1c3602f7e99/setup.py", line 43, in run
          install_dep("{:}_libs".format(tensorrt_module))
        File "/private/var/folders/_b/ldf_nfdx3qndz_5lvxxkchsc0000gn/T/pip-install-8nnh031o/tensorrt_2ea76508e3b64219b3a1a1c3602f7e99/setup.py", line 41, in install_dep
          status.check_returncode()
        File "/Users/ddelange/.pyenv/versions/3.8.10/lib/python3.8/subprocess.py", line 448, in check_returncode
          raise CalledProcessError(self.returncode, self.args, self.stdout,
      subprocess.CalledProcessError: Command '['/Users/ddelange/.pyenv/versions/3.8.10/envs/vv/bin/python3.8', '-m', 'pip', 'install', 'tensorrt_libs==8.6.1', '--index-url', 'https://pypi.nvidia.com']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tensorrt
  Running setup.py clean for tensorrt
Failed to build tensorrt
Installing collected packages: tensorrt
  Running setup.py install for tensorrt ... error
  error: subprocess-exited-with-error

  × Running setup.py install for tensorrt did not run successfully.
  │ exit code: 1
  ╰─> [27 lines of output]
      running install
      /Users/ddelange/.pyenv/versions/3.8.10/envs/vv/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      Looking in indexes: https://pypi.nvidia.com
      ERROR: Could not find a version that satisfies the requirement tensorrt_libs==8.6.1 (from versions: none)
      ERROR: No matching distribution found for tensorrt_libs==8.6.1
      WARNING: There was an error checking the latest version of pip.
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/_b/ldf_nfdx3qndz_5lvxxkchsc0000gn/T/pip-install-8nnh031o/tensorrt_2ea76508e3b64219b3a1a1c3602f7e99/setup.py", line 49, in <module>
          setup(
        File "/Users/ddelange/.pyenv/versions/3.8.10/envs/vv/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/Users/ddelange/.pyenv/versions/3.8.10/lib/python3.8/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/Users/ddelange/.pyenv/versions/3.8.10/lib/python3.8/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/Users/ddelange/.pyenv/versions/3.8.10/lib/python3.8/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/private/var/folders/_b/ldf_nfdx3qndz_5lvxxkchsc0000gn/T/pip-install-8nnh031o/tensorrt_2ea76508e3b64219b3a1a1c3602f7e99/setup.py", line 43, in run
          install_dep("{:}_libs".format(tensorrt_module))
        File "/private/var/folders/_b/ldf_nfdx3qndz_5lvxxkchsc0000gn/T/pip-install-8nnh031o/tensorrt_2ea76508e3b64219b3a1a1c3602f7e99/setup.py", line 41, in install_dep
          status.check_returncode()
        File "/Users/ddelange/.pyenv/versions/3.8.10/lib/python3.8/subprocess.py", line 448, in check_returncode
          raise CalledProcessError(self.returncode, self.args, self.stdout,
      subprocess.CalledProcessError: Command '['/Users/ddelange/.pyenv/versions/3.8.10/envs/vv/bin/python3.8', '-m', 'pip', 'install', 'tensorrt_libs==8.6.1', '--index-url', 'https://pypi.nvidia.com']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> tensorrt

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

[notice] A new release of pip available: 22.3.1 -> 23.1.2
[notice] To update, run: python3.8 -m pip install --upgrade pip

Have you tried the latest release?: 8.6.1 at time of writing

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): N/A

[ddelange]

A small side-request: could you also build wheels for python 3.11 (latest stable, in your scripts PYTHON_MINOR_VERSION=11)?

Many thanks!

[zeroepoch]

Hi @ddelange,
Can you try installing the tensorrt_libs module manually to help troubleshoot the issue? You should see something like the following.

$ python3 -m venv test_venv
$ source test_venv/bin/activate
(test_venv) $ python3 -m pip install --extra-index-url https://pypi.nvidia.com tensorrt_libs
Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com
Collecting tensorrt_libs
  Downloading https://pypi.nvidia.com/tensorrt-libs/tensorrt_libs-8.6.1-py2.py3-none-manylinux_2_17_x86_64.whl (824.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 824.8/824.8 MB 3.1 MB/s eta 0:00:00
Collecting nvidia-cuda-runtime-cu12
  Downloading https://pypi.nvidia.com/nvidia-cuda-runtime-cu12/nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 823.6/823.6 KB 72.6 MB/s eta 0:00:00
Collecting nvidia-cudnn-cu12
  Downloading https://pypi.nvidia.com/nvidia-cudnn-cu12/nvidia_cudnn_cu12-8.9.1.23-py3-none-manylinux1_x86_64.whl (731.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 731.5/731.5 MB 3.5 MB/s eta 0:00:00
Collecting nvidia-cublas-cu12
  Downloading https://pypi.nvidia.com/nvidia-cublas-cu12/nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.6/410.6 MB 6.0 MB/s eta 0:00:00
Installing collected packages: nvidia-cuda-runtime-cu12, nvidia-cublas-cu12, nvidia-cudnn-cu12, tensorrt_libs
Successfully installed nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.1.23 tensorrt_libs-8.6.1

You aren't able to list the packages in your browser at this time on pypi.nvidia.com, so as an additional test you can try downloading the wheel file directly from one of the links in the output above.

[ddelange]

Hi @zeroepoch 👋

Thanks for the quick reply, sounds and looks like everything is as intended on your side.

I'm on macosx_10_9_x86_64 and there are probably no wheels for my platform in your index, in which case it's expected that pip returns (from versions: none).

Will close as won't fix, thanks again!

[HarryCaveMan]

The answer only works if the issue is that you are installing a linux wheel on mac os. I am getting same error using nvidia/cuda:11.3.0-base-ubuntu20.04

I even made sure to try and install the python3 wheel for linux directly:
https://pypi.nvidia.com/tensorrt-libs/tensorrt_libs-8.6.1-py2.py3-none-manylinux_2_17_x86_64.whl

[zeroepoch]

The answer only works if the issue is that you are installing a linux wheel on mac os. I am getting same error using nvidia/cuda:11.3.0-base-ubuntu20.04

I even made sure to try and install the python3 wheel for linux directly: https://pypi.nvidia.com/tensorrt-libs/tensorrt_libs-8.6.1-py2.py3-none-manylinux_2_17_x86_64.whl

@HarryCaveMan can you provide the commands you tried and the errors that were printed? We had some internal users report a similar issue when 8.6.1 was first posted, but I think it got resolved on the server side maybe since I haven't heard any more issues since then.

One user with an old Ubuntu version had to first update setuptools before the internal pip install worked. Can you try python3 -m pip install --upgrade setuptools and see if that helps?

[GoncaloR00]

I had to upgrade pip to be able to install the tensorrt package:
pip3 install --upgrade pip

[zeroepoch]

Thanks @GoncaloR00 for the feedback. You're right it was pip not setuptools that needed to be updated, but updating both wouldn't be a bad idea.

[ddelange]

interesting, getting a similar error now:

• tensorrt install errors with /home/ci/opt/venv/bin/python3: No module named pip
  • command ['/home/ci/opt/venv/bin/python3', '-m', 'pip', 'install', 'tensorrt_libs==8.6.1', '--index-url', 'https://pypi.nvidia.com/']
• yet autogluon ci runs python3 -m pip install --upgrade pip
  • which upgrades /home/ci/opt/venv/lib/python3.8/site-packages

[yunseung-dable]

same problem...
problem solved!
the key is downgrading pip

pip install pip==21.3.1

[ddelange]

Hi @yunseung-dable 👋

Can you upload your error logs.txt to #3078 ?

[ddelange]

you can also try running

TensorRT/python/packaging/frontend_sdist/setup.py

Lines 17 to 40 in 5334f0d

	import platform
	import sys
	from setuptools import setup
	tensorrt_module = "##TENSORRT_MODULE##"
	tensorrt_version = "##TENSORRT_PYTHON_VERSION##"
	index_url = "https://pypi.nvidia.com"
	# cherry-pick information from `packaging.markers.default_environment()` needed to find the right wheel
	# https://github.com/pypa/packaging/blob/23.1/src/packaging/markers.py#L175-L190
	if sys.platform == "linux":
	platform_marker = "manylinux_2_17"
	else:
	raise RuntimeError("TensorRT currently only builds wheels for linux")
	if sys.implementation.name == "cpython":
	implementation_marker = "cp{}".format("".join(platform.python_version_tuple()[:2]))
	else:
	raise RuntimeError("TensorRT currently only builds wheels for CPython")
	machine_marker = platform.machine()
	if machine_marker != "x86_64":
	raise RuntimeError("TensorRT currently only builds wheels for x86_64 processors")

on your system and see if it raises an exception

[couzhei]

I have the same problem here, my pip install tensorrt doesn't even begin to start, it just get frozen at beginning building wheels:

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting tensorrt==8.6.1
  Downloading tensorrt-8.6.1.tar.gz (16 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: tensorrt
  Building wheel for tensorrt (setup.py) ... /

[zeroepoch]

@couzhei, could you maybe try adding -v to the pip install command and see if more gets printed?

[subjectnull]

@couzhei In my case it was silently downloading ~2GB in wheel files, so it wasn't really frozen. Give @zeroepoch 's suggestion a go, you might discover the same!

[couzhei]

@zeroepoch, Thank you! I completely forgot about that option!! With the help of developer.nvidia.com, I managed to successfully add TensorRT to a functioning TensorFlow environment. Though I haven't noticed any performance improvement yet, I assume this because I am no longer seeing the "not finding TensorRT" warning when I import TensorFlow into my projects.

@subjectnull, you're probably right. However, in my five years of using Python day-to-day, I've never encountered such behavior before. Whenever I want to make the installation "silent", I use the -Uqq option in the pip install statement! The thing is, while delving into the details of manually installing TensorRT 8.6.1, I noticed that for some installations, there's a dependency on Cudnn version 8.9.0 as well (which, in my case, required downgrading from 8.9.1 to that version). I always handle the cuda driver (cuda toolkit) and cudnn manually. I wonder if pip install could actually be a hassle-free way of configuring everything with all these details!

[zeroepoch]

The pip installation is designed to be standalone. All you need is a driver install before doing pip to get a full working environment. This is course is assuming the goal is only running TensorRT Python bindings and not C++ development.

[krishac1]

I was initially having the same error message as @ddelange, and then I updated both pip and setuptools. However, now when running the command pip install tensorrt, I get the error after 65 lines of output that there is No Module named pip. Could anyone help me figure out how to troubleshoot this?

[krishac1]

After examining a different thread: github.com//issues/3060 it appears that PiPy, indicated by the following error message:

Looking in indexes: https://pypi.nvidia.com
      ERROR: Could not find a version that satisfies the requirement tensorrt_libs==8.6.1 (from versions: none)
      ERROR: No matching distribution found for tensorrt_libs==8.6.1

cannot host Windows wheels yet.
The resolution seems to be to use the 3.2.4 Zip File Installation section on the Developer page, if you are on Windows.

[zeroepoch]

@krishac1,
Windows wheels are planned for a future release. Although, since Windows wheels are not available today on PyPI what you're seeing would be expected.

[krishac1]

@zeroepoch Thanks! I figured.

[ddelange]

No Module named pip

we also saw this. turned out it was a 'broken' virtual environment where pip symlink didn't exist, only pip3 was available. See also this Dockerfile as reference for a working virtual environment.

[krishac1]

@ddelange I realized I had a faulty venv issue as well - you're right, pip3 was also the one I ended up using. The core of my issue was the Windows incompatibility, however. Thanks for your help.

[laclouis5]

same problem... problem solved! the key is downgrading pip

pip install pip==21.3.1

Had the same error on Ubuntu 20.04 with Python 3.10 and pip 23.2.1. This does not occur with pip 23.0.1.

[zeroepoch]

@laclouis5, you can also pip install wheel first as another way to resolve this issue when you have pip 23.

[Samhayne]

edit: [Solved - See below]

Anyone so kind to give a poor Windows user a hint?

Python 3.11 and

pip install --extra-index-url https://pypi.nvidia.com tensorrt-libs

will give me:

Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com
Collecting tensorrt-libs
  Using cached tensorrt-libs-8.6.1.tar.gz (6.8 kB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'D:\Python\Python311\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Samhayne\\AppData\\Local\\Temp\\pip-install-tkxmv8xe\\tensorrt-libs_454e03b7bfb443408d87c2e2b512cda7\\setup.py'"'"'; __file__='"'"'C:\\Users\\Samhayne\\AppData\\Local\\Temp\\pip-install-tkxmv8xe\\tensorrt-libs_454e03b7bfb443408d87c2e2b512cda7\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\Samhayne\AppData\Local\Temp\pip-pip-egg-info-fuzvb0d_'
       cwd: C:\Users\Samhayne\AppData\Local\Temp\pip-install-tkxmv8xe\tensorrt-libs_454e03b7bfb443408d87c2e2b512cda7\
  Complete output (15 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "C:\Users\Samhayne\AppData\Local\Temp\pip-install-tkxmv8xe\tensorrt-libs_454e03b7bfb443408d87c2e2b512cda7\setup.py", line 137, in <module>
      raise RuntimeError(open("ERROR.txt", "r").read())
  RuntimeError:
  ###########################################################################################
  The package you are trying to install is only a placeholder project on PyPI.org repository.
  This package is hosted on NVIDIA Python Package Index.

  This package can be installed as:
  ```
  $ pip install --extra-index-url https://pypi.nvidia.com tensorrt-libs
  ```
  ###########################################################################################

  ----------------------------------------

I also failed when trying to install an older version...
pip install tensorrt==8.6.0
ERROR: No matching distribution found for tensorrt==8.6.0
... but maybe I did it wrong.

Downgrading pip to version 21.3.1 didn't help either.

I'm totally lost.

=============
edit:
Nevermind @krishac1 mentioned the solution in #2933 (comment)
I just didn't understand it.
Installing manually as described here works:
https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-zip

[ddelange]

@zeroepoch it looks like the RuntimeError message raised by tensorrt_libs on official PyPI needs to additionally mention --no-cache-dir, otherwise the second run (from those instructions) will not trigger a download from the NVIDIA index

it might also be nice to mention the env var setup analogous to #3080 in that RuntimeError message and on the official PyPI readme

[zeroepoch]

@Samhayne,
We don't have Windows wheels available that can be installed from PyPI.org. That's why you have to unpack the library using the zip and install just the bindings wheel rather than these standalone wheels.

[zeroepoch]

@ddelange,
That fake package, which is just a placeholder to guard the project, is a generic package uploaded for all NVIDIA packages released using the internal tools we use. I can see how this can be a problem in this scenario so let me open an internal bug on that team to start the discussion on this issue.

[zeroepoch]

@ddelange, the fake packages recently published for the TRT 9.0 pre-release wheels now have --no-cache-dir in the help description.

[joeyballentine]

@zeroepoch
sorry for necroposting (and for being a bit direct), but why can't you guys just publish the Windows wheels that you have available in the zip onto pypi? It seems kinda ridiculous to me that this isn't a solved problem already. What's holding you guys back?

[ddelange]

@joeyballentine I guess the thread in #3080 is a pretty good summary

[zeroepoch]

@joeyballentine, we do actually have some Windows wheels published, but they are currently pre-release. You can see one example component of the full package at https://pypi.nvidia.com/tensorrt-libs/. The next general release (supporting both LLM and non-LLM networks and the full scope of GPUs) will include Windows wheels not under a pre-release (--pre) flag.

The reason we don't release the wheel files in the zip file is because they don't contain the libraries inside, which means they are not standalone. The release linked above is a full release (with libs), but TRT 9.x is specifically for LLM support and tested on mostly data center GPUs. In summary things will improve and become more seamless, but you'll need to wait for the next major TRT release.