failed to install onnxruntime-gpu on Jetson Nano with the latest image (Jetpack 4.4.1) #6124

guybartal · 2020-12-13T20:48:55Z

Describe the bug
failed to install onnxruntime-gpu PyPi package on Jetson Nano device with the latest image (Jetpack 4.4.1)

Urgency
ASAP

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux gubert-jetson-ha 4.9.140-tegra Set up CI with Azure Pipelines #1 SMP PREEMPT Tue Oct 27 21:02:37 PDT 2020 aarch64 aarch64 aarch64 GNU/Linux
ONNX Runtime installed from (source or binary): pip3 install onnxruntime-gpu==1.4.0
ONNX Runtime version: 1.4.0
Python version: 3.6.9
Visual Studio version (if applicable):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: CUDA Version 10.2.89
GPU model and memory: Jetson Nano 4GB

To Reproduce

Describe steps/code to reproduce the behavior.
Attach the ONNX model to the issue (where applicable) to expedite investigation.

pip3 install onnxruntime-gpu
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement onnxruntime-gpu
ERROR: No matching distribution found for onnxruntime-gpu

Expected behavior
A clear and concise description of what you expected to happen.
the package should be installed as onnxruntime==1.4.0 does successfully

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here. If the issue is about a particular model, please share the model details as well to facilitate debugging.

The text was updated successfully, but these errors were encountered:

guybartal · 2020-12-13T21:37:01Z

found and answer how to install here:
https://elinux.org/Jetson_Zoo#ONNX_Runtime

snnn · 2020-12-14T18:26:35Z

aarch64 gpu? We don't have such a package yet.

jywu-msft · 2020-12-28T19:28:52Z

Please note that we have ORT 1.6.0 GPU python (3.6, 3.7, 3.8) packages for Jetpack 4.4/4.4.1 now!
https://elinux.org/Jetson_Zoo#ONNX_Runtime

CalebBates · 2021-06-18T17:45:57Z

Hello, I am getting the following error:

ModuleNotFoundError: No module named 'Jetson'

I try to run the following command:
sudo pip3 install Jetson

I then get the following 2 errors:

Could not find a version that satisfies the requirement Jetson (from versions: none)
No matching distribution found for Jetson

What could be the issue?

Thank you for the help!

jywu-msft · 2021-06-18T17:54:14Z

Hello, I am getting the following error:

ModuleNotFoundError: No module named 'Jetson'

I try to run the following command:
sudo pip3 install Jetson

I then get the following 2 errors:

Could not find a version that satisfies the requirement Jetson (from versions: none)

No matching distribution found for Jetson

What could be the issue?

Thank you for the help!

there's no package named Jetson.
follow the instructions at https://elinux.org/Jetson_Zoo#ONNX_Runtime

i.e.

Download pip wheel from location mentioned above

$ wget https://nvidia.box.com/s/bfs688apyvor4eo8sf3y1oqtnarwafww -O onnxruntime_gpu-1.8.0-cp36-cp36m-linux_aarch64.whl

Install pip wheel

$ pip3 install onnxruntime_gpu-1.8.0-cp36-cp36m-linux_aarch64.whl

CalebBates · 2021-06-18T18:02:05Z

To clarify, run the following two commands?

Download pip wheel from location mentioned above

$ wget https://nvidia.box.com/s/bfs688apyvor4eo8sf3y1oqtnarwafww -O onnxruntime_gpu-1.8.0-cp36-cp36m-linux_aarch64.whl

Install pip wheel

$ pip3 install onnxruntime_gpu-1.8.0-cp36-cp36m-linux_aarch64.whl

everdrone · 2021-08-19T09:22:58Z

Still getting an error with the .whl file, tried both on python 3.9.6, on a Jetson Nano 4G

ERROR: onnxruntime_gpu-1.8.0-cp36-cp36m-linux_aarch64.whl is not a supported wheel on this platform.

weosk · 2021-12-07T18:21:29Z

Try following the link jywu-msft posted: https://elinux.org/Jetson_Zoo#ONNX_Runtime and download the pip-wheel appropriate to the python version (python3 --version) you are using and run the pip3 install onnxruntim... command. Worked for me on a Jetson Nano.

dav-ell · 2022-04-15T14:24:05Z

@weosk's method worked for me, but I'm getting an error trying to import the library:

import onnxruntime as ort

gives

ModuleNotFoundError: No module named 'onnxruntime'

Is there a different way to import?

patrick620 · 2022-04-19T05:13:14Z

@dav-ell Hi, I can install onnxruntime-gpu 1.10.0 on my nano with Jetpack version 4.5-b129

# Download pip wheel from location mentioned above
wget https://nvidia.box.com/shared/static/jy7nqva7l88mq9i8bw3g3sklzf4kccn2.whl -O onnxruntime_gpu-1.10.0-cp36-cp36m-linux_aarch64.whl

Use this command to install .whl

python3 -m pip install onnxruntime_gpu-1.10.0-cp36-cp36m-linux_aarch64.whl

Instead of pip3 install onnxruntime_gpu-1.10.0-cp36-cp36m-linux_aarch64.whl

dav-ell · 2022-04-23T18:49:48Z

Thanks @patrick620! That did the trick. Fyi to those interested, TensorrtExecutionProvider is not supported using the above method.

emay12 · 2022-06-21T18:05:35Z

im trying to install onnxruntime on my jetson nano but im running into an issue where im installing like above, but protobuf wont install causing the entire install to fail.

the error im getting is :
protobuf requires Python '>=3.7' but the running python is 3.6.9

and correct me if im wrong, but jetpack needs python3.6 ? so how do i get around this error? thanks!

jywu-msft · 2022-06-21T19:07:46Z

im trying to install onnxruntime on my jetson nano but im running into an issue where im installing like above, but protobuf wont install causing the entire install to fail.

the error im getting is : protobuf requires Python '>=3.7' but the running python is 3.6.9

and correct me if im wrong, but jetpack needs python3.6 ? so how do i get around this error? thanks!

you can install a different version of python on the jetson.
e.g. apt install python3.7

jywu-msft · 2022-06-21T19:11:31Z

Thanks @patrick620! That did the trick. Fyi to those interested, TensorrtExecutionProvider is not supported using the above method.

Can you elaborate on what you mean by TensorrtExecutionProvider is not supported using above method?
Starting with version 1.10 onnxruntime gpu package should include TensorrtExecutionProvider

dlangerm · 2022-06-28T14:01:51Z

im trying to install onnxruntime on my jetson nano but im running into an issue where im installing like above, but protobuf wont install causing the entire install to fail.
the error im getting is : protobuf requires Python '>=3.7' but the running python is 3.6.9
and correct me if im wrong, but jetpack needs python3.6 ? so how do i get around this error? thanks!

you can install a different version of python on the jetson. e.g. apt install python3.7

I just ran into this as well, the wheel referenced in the docs seems to be broken

LeoYong95 · 2023-03-06T04:15:11Z

I am running on Jetpack 5.1 on Jetson Xavier NX I tried the Jetpack 5.0 version pip install in here but it does'nt work. is there a supported version for Jetpack 5.1?

ErenCaldiran · 2023-04-28T05:49:30Z

I also can't run onnxruntime-gpu for Jetpack 5.1.1 on Jetson orin nano developer kit. Onnxruntime-gpu 1.12.1 for Jetpack 5.0 does not see the gpu and only works for cpu,

storm12t48 · 2023-05-12T11:10:10Z

ou dans le kit de développement nano. Onnxruntime-gpu 1.12.1 pour Jetpack 5.0 ne voit pas le gpu et ne fonctionne que pour le cpu,

+1 for Jetson AGX orin on Jetpack 5.1.1 with 1.12.1 dont work , do you have new version?

zachary-zheng · 2023-07-08T01:05:35Z

same here, Jetpack 5.1, do you have solution?

jywu-msft · 2023-07-08T23:29:48Z

FYI,
@yf711 is working on creating a package that supports JetPack 5.1.1 which will get published to the Jetson Zoo

lweingart · 2023-10-16T18:23:55Z

Hi everyone,

I have a Jetpack 5.1 and python 3.8 on a Jetson AGX Orin, but everything I try keeps failing.
Following some previous comments, here are my attempts:

$ pip3 install onnxruntime_gpu-1.15.1-cp38-cp38m-linux_aarch64.whl
Defaulting to user installation because normal site-packages is not writeable
ERROR: onnxruntime_gpu-1.15.1-cp38-cp38m-linux_aarch64.whl is not a supported wheel on this platform.
WARNING: There was an error checking the latest version of pip.

$ python3 -m pip install onnxruntime_gpu-1.15.1-cp38-cp38m-linux_aarch64.whl
Defaulting to user installation because normal site-packages is not writeable
ERROR: onnxruntime_gpu-1.15.1-cp38-cp38m-linux_aarch64.whl is not a supported wheel on this platform.
WARNING: There was an error checking the latest version of pip.

$ pip3 install onnxruntime_gpu-1.16.0-cp38-cp38m-linux_aarch64.whl
Defaulting to user installation because normal site-packages is not writeable
ERROR: onnxruntime_gpu-1.16.0-cp38-cp38m-linux_aarch64.whl is not a supported wheel on this platform.
WARNING: There was an error checking the latest version of pip.

$ python3 -m pip install onnxruntime_gpu-1.16.0-cp38-cp38m-linux_aarch64.whl
Defaulting to user installation because normal site-packages is not writeable
ERROR: onnxruntime_gpu-1.16.0-cp38-cp38m-linux_aarch64.whl is not a supported wheel on this platform.
WARNING: There was an error checking the latest version of pip.

I'm pretty sure I tried everything, downloaded the right version of files for my Jetson install, but keep failing.
Any idea what could help ?

Thank you very much in advance for your help
Cheers

dav-ell · 2023-10-16T21:56:34Z

Looks like you downloaded the wrong version of the wheel. Make sure you download one with a "cp38" on it. The 36/38 refers to Python 3.6/3.8.

…

On Mon, Oct 16, 2023, 2:24 PM Laurent Weingart ***@***.***> wrote: Hi everyone, I have a Jetpack 5.1 and python 3.8 on a Jetson AGX Orin, but everything I try keeps failing. Following some previous comments, here are my attempts: $ pip3 install onnxruntime_gpu-1.15.1-cp36-cp36m-linux_aarch64.whl Defaulting to user installation because normal site-packages is not writeable ERROR: onnxruntime_gpu-1.15.1-cp36-cp36m-linux_aarch64.whl is not a supported wheel on this platform. WARNING: There was an error checking the latest version of pip. $ python3 -m pip install onnxruntime_gpu-1.15.1-cp36-cp36m-linux_aarch64.whl Defaulting to user installation because normal site-packages is not writeable ERROR: onnxruntime_gpu-1.15.1-cp36-cp36m-linux_aarch64.whl is not a supported wheel on this platform. WARNING: There was an error checking the latest version of pip. $ pip3 install onnxruntime_gpu-1.16.0-cp36-cp36m-linux_aarch64.whl Defaulting to user installation because normal site-packages is not writeable ERROR: onnxruntime_gpu-1.16.0-cp36-cp36m-linux_aarch64.whl is not a supported wheel on this platform. WARNING: There was an error checking the latest version of pip. $ python3 -m pip install onnxruntime_gpu-1.16.0-cp36-cp36m-linux_aarch64.whl Defaulting to user installation because normal site-packages is not writeable ERROR: onnxruntime_gpu-1.16.0-cp36-cp36m-linux_aarch64.whl is not a supported wheel on this platform. WARNING: There was an error checking the latest version of pip. I'm pretty sure I tried everything, downloaded the right version of files for my Jetson install, but keep failing. Any idea what could help ? Thank you very much in advance for your help Cheers — Reply to this email directly, view it on GitHub <#6124 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEJHJXLLDE5WT6WLWSNK7R3X7V3UPAVCNFSM4UZY4CC2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZWGUYDKMRVHAYQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

lweingart · 2023-10-17T10:26:47Z

Hello dav-ell,

Actually you just made me understand the error I made, which is actually different from what could be thought at first.
I actually copied the link from another computer (from which I SSH to my Jetson) and the file is named https://nvidia.box.com/shared/static/iizg3ggrtdkqawkmebbfixo7sce6j365.whl
I used wget with this link and renamed the resulting file myself into onnxruntime_gpu-1.16.0-cp36-cp36m-linux_aarch64.whl or onnxruntime_gpu-1.16.0-cp38-cp38m-linux_aarch64.whl

Now I actually downloaded the file from my own computer and the result is indeed onnxruntime_gpu-1.16.0-cp38-cp38-linux_aarch64.whl
I SCP'd it to my Jetson and the install went fine this time.

Thank you very much for your help!

Cheers

lweingart · 2023-10-21T14:16:35Z

Hi again,

I have a question, hopefully you can help me.
I have my Jetson AGX Orin with Jetpack 5.1 and after successfully installing onnxruntime-gpu, it still only uses the CPU, as shown by the following:

# jetson @ jetson-orin in ~/Downloads [16:11:28]
$ pip3 install -U onnxruntime_gpu-1.16.0-cp38-cp38-linux_aarch64.whl
Defaulting to user installation because normal site-packages is not writeable
Processing ./onnxruntime_gpu-1.16.0-cp38-cp38-linux_aarch64.whl
Requirement already satisfied: coloredlogs in /home/jetson/.local/lib/python3.8/site-packages (from onnxruntime-gpu==1.16.0) (15.0.1)
Requirement already satisfied: flatbuffers in /home/jetson/.local/lib/python3.8/site-packages (from onnxruntime-gpu==1.16.0) (2.0.7)
Requirement already satisfied: numpy>=1.24.4 in /home/jetson/.local/lib/python3.8/site-packages (from onnxruntime-gpu==1.16.0) (1.24.4)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from onnxruntime-gpu==1.16.0) (23.0)
Requirement already satisfied: protobuf in /home/jetson/.local/lib/python3.8/site-packages (from onnxruntime-gpu==1.16.0) (3.20.3)
Requirement already satisfied: sympy in /home/jetson/.local/lib/python3.8/site-packages (from onnxruntime-gpu==1.16.0) (1.11.1)
Requirement already satisfied: humanfriendly>=9.1 in /home/jetson/.local/lib/python3.8/site-packages (from coloredlogs->onnxruntime-gpu==1.16.0) (10.0)
Requirement already satisfied: mpmath>=0.19 in /home/jetson/.local/lib/python3.8/site-packages (from sympy->onnxruntime-gpu==1.16.0) (1.2.1)
onnxruntime-gpu is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.
WARNING: There was an error checking the latest version of pip.

# jetson @ jetson-orin in ~/Downloads [16:11:46]
$ python3
Python 3.8.10 (default, May 26 2023, 14:05:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import onnxruntime as ort
>>> ort.get_device()
'CPU'
>>>

Is there an additional configuration to perform in order to ensure that onnx will use the GPU ?

Thank you very much for your help in advance

EDIT:
I found out that there is a compatibility issue to take into account.
onnxruntime 1.15 works with CUDA 11.8 and cuDNN 8.2.4 (on Linux) as explained here

It also mentions that it has been tested with CUDA versions from 11.6 up to 11.8, and cuDNN from 8.2.4 up to 8.7.0

I have CUDA 11.4 and cuDNN 8.6 that came with Jetpack 5.1
Is there a way to have the right version of everything at the same time on my Jetson ?

I fail to see a working onnxruntime-gpu version for CUDA 11.4, cuDNN 8.6 and Jetpack 5.1

Any help or advice would be greatly appreciated :-)

EDIT 2:
It appears that it's now possible to upgrade CUDA on Jetson devices without changing the jetpack version.
I wonder, could you please share which versions of CUDA and cuDNN are working with onnxruntime-gpu 1.16.0?

I see that the latest CUDA version that can be installed on Jetson is currently the 12.2 but I fail to see the list of compatibilities for onnxruntime-gpu 1.6

Thank you in advance :-)

Cheers

lweingart · 2023-10-22T11:12:36Z

Hi again,

I reinstalled CUDA 11.8 and it doesn't seem to get any better:

# jetson @ jetson-orin in ~/git/jetsonUtilities on git:master o [13:07:07]
$ python3 jetsonInfo.py
NVIDIA Jetson AGX Orin
 L4T 35.2.1 [ JetPack 5.1 ]
   Ubuntu 20.04.6 LTS
   Kernel Version: 5.10.104-tegra
 CUDA 11.8.89
   CUDA Architecture: 8.7
 OpenCV version: 4.5.4
   OpenCV Cuda: YES
 CUDNN: 8.6.0.166
 TensorRT: 8.5.2.2
 Vision Works: NOT_INSTALLED
 VPI: 2.2.4
 Vulcan: 1.3.204

# jetson @ jetson-orin in ~/git/jetsonUtilities on git:master o [13:07:49]
$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:43:33_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

# jetson @ jetson-orin in ~ [13:07:58]
$ python3
Python 3.8.10 (default, May 26 2023, 14:05:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import onnxruntime as ort
>>> ort.__version__
'1.15.1'
>>> ort.get_device()
'CPU'
>>>

Would any of you have an idea as to what I need to do for my onnx runtime environment to use the GPU?
Thank you very much for any help you could send my way.

Cheers

jywu-msft · 2023-10-23T02:50:48Z

did you previously install onnxruntime?
the error message "onnxruntime-gpu is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel." seems to indicate so.
please do pip uninstall onnxruntime and pip uninstall onnxruntime-gpu
and reinstall the onnxruntime-gpu package you downloaded from jetson zoo.

lweingart · 2023-10-23T07:29:06Z

Hello jywu,

Thank you for spotting this, I installed the 1.15.1 and 1.16 versions from the downloaded wheel many times and I didn't give the necessary attention to that line.

My issue is indeed fixed now and I finally see the GPU as device :-)
Thank you very much for your help

Cheers

snnn added api:Python build build issues; typically submitted using template feature request request for unsupported feature or enhancement labels Dec 14, 2020

ashbhandare mentioned this issue May 9, 2022

Build fail on Jetson Nano #11462

Closed

iperezx mentioned this issue May 13, 2022

Onnnx model needs to be ran on the gpu and not the cpu. iperezx/sage-smoke-detection#5

Open

sophies927 added api issues related to all other APIs: C, C++, Python, etc. and removed api:Python labels Aug 12, 2022

snnn closed this as completed Oct 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

failed to install onnxruntime-gpu on Jetson Nano with the latest image (Jetpack 4.4.1) #6124

failed to install onnxruntime-gpu on Jetson Nano with the latest image (Jetpack 4.4.1) #6124

guybartal commented Dec 13, 2020

guybartal commented Dec 13, 2020

snnn commented Dec 14, 2020

jywu-msft commented Dec 28, 2020

CalebBates commented Jun 18, 2021

jywu-msft commented Jun 18, 2021

CalebBates commented Jun 18, 2021

everdrone commented Aug 19, 2021 •

edited

weosk commented Dec 7, 2021 •

edited

dav-ell commented Apr 15, 2022

patrick620 commented Apr 19, 2022

dav-ell commented Apr 23, 2022

emay12 commented Jun 21, 2022

jywu-msft commented Jun 21, 2022

jywu-msft commented Jun 21, 2022

dlangerm commented Jun 28, 2022

LeoYong95 commented Mar 6, 2023

ErenCaldiran commented Apr 28, 2023

storm12t48 commented May 12, 2023 •

edited

zachary-zheng commented Jul 8, 2023

jywu-msft commented Jul 8, 2023

lweingart commented Oct 16, 2023 •

edited

dav-ell commented Oct 16, 2023 via email

lweingart commented Oct 17, 2023

lweingart commented Oct 21, 2023 •

edited

lweingart commented Oct 22, 2023

jywu-msft commented Oct 23, 2023

lweingart commented Oct 23, 2023

failed to install onnxruntime-gpu on Jetson Nano with the latest image (Jetpack 4.4.1) #6124

failed to install onnxruntime-gpu on Jetson Nano with the latest image (Jetpack 4.4.1) #6124

Comments

guybartal commented Dec 13, 2020

guybartal commented Dec 13, 2020

snnn commented Dec 14, 2020

jywu-msft commented Dec 28, 2020

CalebBates commented Jun 18, 2021

jywu-msft commented Jun 18, 2021

Download pip wheel from location mentioned above

Install pip wheel

CalebBates commented Jun 18, 2021

Download pip wheel from location mentioned above

Install pip wheel

everdrone commented Aug 19, 2021 • edited

weosk commented Dec 7, 2021 • edited

dav-ell commented Apr 15, 2022

patrick620 commented Apr 19, 2022

dav-ell commented Apr 23, 2022

emay12 commented Jun 21, 2022

jywu-msft commented Jun 21, 2022

jywu-msft commented Jun 21, 2022

dlangerm commented Jun 28, 2022

LeoYong95 commented Mar 6, 2023

ErenCaldiran commented Apr 28, 2023

storm12t48 commented May 12, 2023 • edited

zachary-zheng commented Jul 8, 2023

jywu-msft commented Jul 8, 2023

lweingart commented Oct 16, 2023 • edited

dav-ell commented Oct 16, 2023 via email

lweingart commented Oct 17, 2023

lweingart commented Oct 21, 2023 • edited

lweingart commented Oct 22, 2023

jywu-msft commented Oct 23, 2023

lweingart commented Oct 23, 2023

everdrone commented Aug 19, 2021 •

edited

weosk commented Dec 7, 2021 •

edited

storm12t48 commented May 12, 2023 •

edited

lweingart commented Oct 16, 2023 •

edited

lweingart commented Oct 21, 2023 •

edited