Python 3.10 and 3.11 support?

[baudneo]

Anyone compiled pycoral for 3.10 and 3.11? I am attempting to build them but I was wondering if someone else has already done this?

[fenil1608]

Hi, any update on this ? I'm trying to use pycoral on ubuntu 22 with python 3.10.

[baudneo]

I'm assuming no since the assignee hasn't replied to this issue but did reply to a newer issue.

It seems maybe it's based on dependencies that aren't available in 3.10+ yet. Only option I can see is the community forking the project and attempting to build for 3.10+.

I will give it a try when I have a bit of spare time.

[travisariggs]

@baudneo , did you ever find any spare time to try building for 3.10? I am very interested in running on 3.10. I tried simply updating the build.sh docker image to use ubuntu:22.04, but that wasn't enough to get it to work.

[baudneo]

It's based on wheels for scikit or tensorflow from what I understand. When the upstream team releases the wheels than 3.x integration can happen

[peteh]

As far as I understand both scikit and tensorflow have 3.10 compatibility marked in pipy. However, I'm failing to run the examples on ubuntu 22.04

[baudneo]

Compatibility yes, wheels no. Wheels are needed by pycoral to build their 3.10 stuff.

[hjonnala]

please try the wheels, built locally on Linux, for pyhton3.10 and let us know if those are working on ubuntu:22.04. Thanks!

[baudneo]

Will give her a go tonight!

[baudneo]

Ill have to make a build of Ubuntu 21.10 (glibc 2.34) or 22.04 (glibc 2.35) as 21.04 uses glibc 2.33 and throws this error ->

>>> from pycoral.adapters import common
>>> from pycoral.adapters import common, detect
>>> from pycoral.utils.edgetpu import make_interpreter as make_interpreter, list_edge_tpus
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/tyler/.local/lib/python3.10/site-packages/pycoral/utils/edgetpu.py", line 24, in <module>
    from pycoral.pybind._pywrap_coral import GetRuntimeVersion as get_runtime_version
ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /home/tyler/.local/lib/python3.10/site-packages/pycoral/pybind/_pywrap_coral.cpython-310-x86_64-linux-gnu.so)
>>>

[travisariggs]

I tried it on my yocto kirkstone system with the same error (though different version of GLIBC)

      from pycoral.utils import edgetpu
  File "/opt/ros/noetic/lib/python3.10/site-packages/pycoral/utils/edgetpu.py", line 24, in <module>
      from pycoral.pybind._pywrap_coral import GetRuntimeVersion as get_runtime_version
ImportError: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /opt/ros/noetic/lib/python3.10/site-packages/pycoral/pybind/_pywrap_coral.cpython-310-x86_64-linux-gnu.so)

Do you have instructions to build this shared object?

[hjonnala]

Please install the bazel and try building the pycoral itself and share the complete logs if there are any errors.

git clone --recurse-submodules https://github.com/google-coral/pycoral
cd pycoral
make
make wheel
make tflite-wheel

[travisariggs]

I was not able to build on a fresh image of Ubuntu 22.04.

I installed bazelisk, cloned the pycoral repo and ran make.

My error log is attached.

build-fail.log

[hjonnala]

Please check whether /usr/include/python3.10 path exist and change _get_python_include function in this file as below and rerun the make command: /home/ubuntu/.cache/bazel/_bazel_ubuntu/d8383202a9b6366beeb5d874af8d5d3f/external/org_tensorflow/third_party/py/python_configure.bzl

def _get_python_include(repository_ctx, python_bin):
    """Gets the python include path."""
    return "/usr/include/python3.10"

[hjonnala]

You may get some other errors after rerun. Please add the below lines to /home/ubuntu/.cache/bazel/_bazel_ubuntu/d8383202a9b6366beeb5d874af8d5d3f/external/ruy/ruy/block_map.cc

#include <stdexcept>
#include <limits>

[travisariggs]

The path /usr/include/python3.10 did not exist on my Ubuntu 22 system. I installed "python3-numpy", which did create that base path.

I modified the _get_python_include to return "/usr/include/python3.10". I then ran make. Here is the log of that failure:

20221110-build-fail-1.log

I then added <stdexcept> and <limits> to block_map.cc at the path you gave.

Unfortunately, running make resulted in an error. It was unable to find Python.h. That is captured in this log:

20221110-build-fail-2.log

I searched the filesystem and found that Python.h was not installed. I ran apt install python3-dev, which installed Python.h to /usr/include/python3.10/Python.h.

I ran make again with the same error. It still wasn't able to find Python.h. I tried rebooting, running make clean and then make again...same error.

20221110-build-fail-3.log

Do I need to reset some cached state in bazel?

Also, do I need to run some sort of setup script to install the dependencies that you expect to be present at build time? I certainly needed to install python3-dev, but are there others?

[baudneo]

It has compiled and is working as expected on a new 22.04 Jammy install

[hjonnala]

I ran make again with the same error. It still wasn't able to find Python.h. I tried rebooting, running make clean and then make again...same error.

Can you try sudo apt-get install python3-dev.

If its not working, Please check this path for Python.h: /home/ubuntu/.cache/bazel/_bazel_ubuntu/d8383202a9b6366beeb5d874af8d5d3f/execroot/pycoral/bazel-out/k8-opt/bin/external/local_config_python/python_include/Python.h

If its not there please add that file to pyhton_include folder.

[travisariggs]

I started with a brand new install of Ubuntu 22. This time, I installed these apt packages before attempting to build the wheel files:

1. build-essential
2. git
3. python3-dev
4. python3-wheel
5. python3-numpy

Then, I did the following:

1. Installed bazelisk
2. Cloned pycoral
3. Ran make (failed)
4. Edited python_configure.bzl to return the python include path
5. Ran make (failed)
6. Edited block_make.cc to add the extra headers
7. Ran make
8. Ran make wheel
9. Ran make tflite-wheel

Wheels built successfully!

However, after installing them on my non-Ubuntu (Yocto Kirkstone) system, I get a runtime error involving GLIBCXX:

Traceback (most recent call last):
     from pycoral.utils import edgetpu
   File "/opt/ros/noetic/lib/python3.10/site-packages/pycoral/utils/edgetpu.py", line 24, in <module>
     from pycoral.pybind._pywrap_coral import GetRuntimeVersion as get_runtime_version
 ImportError: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /opt/ros/noetic/lib/python3.10/site-packages/pycoral/pybind/_pywrap_coral.cpython-310-x86_64-linux-gnu.so)

According to this (https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html), it looks like I need to compile with gcc 12.1, but ubuntu 22 defaults to gcc 11.1.

I am going to try installing gcc 12.1 and rebuilding next week.

Does that seem reasonable?

[baudneo]

I started with a brand new install of Ubuntu 22. This time, I installed these apt packages before attempting to build the wheel files:

1. build-essential
2. git
3. python3-dev
4. python3-wheel
5. python3-numpy

Then, I did the following:

1. Installed bazelisk
2. Cloned pycoral
3. Ran make (failed)
4. Edited python_configure.bzl to return the python include path
5. Ran make (failed)
6. Edited block_make.cc to add the extra headers
7. Ran make
8. Ran make wheel
9. Ran make tflite-wheel

Wheels built successfully!

However, after installing them on my non-Ubuntu (Yocto Kirkstone) system, I get a runtime error involving GLIBCXX:

Traceback (most recent call last):
     from pycoral.utils import edgetpu
   File "/opt/ros/noetic/lib/python3.10/site-packages/pycoral/utils/edgetpu.py", line 24, in <module>
     from pycoral.pybind._pywrap_coral import GetRuntimeVersion as get_runtime_version
 ImportError: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /opt/ros/noetic/lib/python3.10/site-packages/pycoral/pybind/_pywrap_coral.cpython-310-x86_64-linux-gnu.so)

According to this (https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html), it looks like I need to compile with gcc 12.1, but ubuntu 22 defaults to gcc 11.1.

I am going to try installing gcc 12.1 and rebuilding next week.

Does that seem reasonable?

Build it on your yocto system using its libs.

[rdejana]

Any updates on this?

[baudneo]

Any updates on this?

Idk if 3.10 has easily installable wheels but you can build it yourself by following the instructions in this thread.

[nandra]

I've tried to add python 3.10 support directly to build all binaries using docker but it's failing on various places. Does anybody work on it please? Thanks.

[jakew009]

Is there any reason why only a very limited subset of wheels for python 3.10 have been released? Is this repo essentially abandoned? Building the wheels from scratch is a nightmare.

[jakew009]

@travisariggs did you ever managed to build a wheel successfully for kirkstone? We are trying to do the same thing

[rgriffogoes]

Just followed the steps above and it works, but ensure that you have an older version of bazel.

I'm on a fresh Ubuntu 22.04 Server, and installing latest Bazel (6.0.0) the process failed (with something like Please use constraints from @platforms repository embedded in Bazel)

Had to purge it + install an old version:

sudo apt-get --purge remove bazel

sudo apt install bazel-5.3.2
sudo ln -s /usr/bin/bazel-5.3.2 /usr/bin/bazel

Then the process described by travisariggs on Nov 11th worked (including the 2 edits in the cache files).

[rikardn]

I managed to build wheels for the most common desktop platforms. Wheels are available from here https://github.com/pharmpy/tflite-runtime-wheels/releases/tag/v2.12.0. The build workflow can be found in the repo under .github/workflows.

I haven't tested them yet on all these platforms.

After these wheels are tested would the maintainers of tflite be open to adding them to PyPI?

[oberluz]

I've created #128 to upgrade pycoral from 2.2.0 to 2.7.0 with support for python 3.10+

Anyone interested in wheels can either:

1. Download them from the releases page: https://github.com/oberluz/libcoral/releases/tag/2.7.0

or

2. Clone my repo https://github.com/oberluz/pycoral recursively and build them using the same methods used in google-coral/pycoral

[David-OConnor]

Hero.

FYI: It looks like those are source code, vice wheels in the link.

[oberluz]

I haven't uploaded them up yet, still bakeing. I'll post here when done in the next hour or so

[baudneo]

Sick, good thing the community took the time to support a multi billion dollar companies hardware.

Thank you @oberluz

Edit: Also, this is a textbook example of why OSS is so important. Once the company looses interest, the community can pick it back up.

[oberluz]

Sick, good thing the community took the time to support a multi billion dollar companies hardware.

Thank you @oberluz

No worries.

If anyone has cloned the repo since i made the announcement please do:

$ git pull
$ git submodules update --init --recursive

as I've added a commit that modifies the libcoral submodule.

I'm rebuilding the wheels and will upload them shortly.

@baudneo you mentioned you where interested in 3.11 which I haven't tested yet. I'll add it shortly using debian bookworm to build it...

[oberluz]

@David-OConnor @baudneo Here are the wheels. Let me know how you go...:

https://github.com/oberluz/pycoral/releases/tag/2.7.0

[David-OConnor]

Love it. Should I be able to build a wheel for Windows using the source versions you posted and PyCoral instructions?

[oberluz]

Love it. Should I be able to build a wheel for Windows using the source versions you posted and PyCoral instructions?

I don't have a windows machine or VM. All I can say is try it. I haven't modified the way it build so if you were able to do it before you should be able to now.

If you do it would be good if we can include the windows wheels in the release for others.

[David-OConnor]

Awesome; I'll post back once I try if it works

[David-OConnor]

On Ubuntu 20, Python 3.11, installing either the pycoral or tfliteruntime wheels results in Error: pycoral...wheel is not a supported wheel on this platform. I wonder if I could build for 3.11 using your sources though.

[David-OConnor]

I attempted to build on Windows/Py3.11, and Ubuntu/Py3.11. Same error on both:

ERROR: Could not find a version that satisfies the requirement tflite-runtime==2.7.0.post1 (from pycoral) (from versions: none)
ERROR: No matching distribution found for tflite-runtime==2.7.0.post1

I wonder if the fact the assets are split into pycoral and tflite_runtime wheels is relevant? Of note, I don't know how this system works, and was previously confused about the difference between the two libs.

[oberluz]

Yes, I'm still adding support for 3.11. I'll need to try the next tensorflow version that has a recent version of pybind11 in its bazel configuration. There is a branch for 3.11 but us failing due to the reasons I've explained Can you try building the wheels for 3.6, 3.7, 3.8, 3.9 and 3.10? Those should work.

…

On Thu, 2 Nov 2023, 23:11 David-OConnor, ***@***.***> wrote: I attempted to build on Windows/Py3.11, and Ubuntu/Py3.11. Same error on both: ERROR: Could not find a version that satisfies the requirement tflite-runtime==2.7.0.post1 (from pycoral) (from versions: none) ERROR: No matching distribution found for tflite-runtime==2.7.0.post1 — Reply to this email directly, view it on GitHub <#85 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKFR7SEOPYF2XEFAOV5UWDYCQLAVAVCNFSM56DDBEI2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZZGE3DGMZYHEYA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[oberluz]

I attempted to build on Windows/Py3.11, and Ubuntu/Py3.11. Same error on both:

ERROR: Could not find a version that satisfies the requirement tflite-runtime==2.7.0.post1 (from pycoral) (from versions: none)
ERROR: No matching distribution found for tflite-runtime==2.7.0.post1

I wonder if the fact the assets are split into pycoral and tflite_runtime wheels is relevant? Of note, I don't know how this system works, and was previously confused about the difference between the two libs.

I guess they split them originally so that tflite-runtime could be used for other things then pycoral.

Regarding your error, you need to install tflite-runtime before pycoral..

[oberluz]

I attempted to build on Windows/Py3.11, and Ubuntu/Py3.11. Same error on both:

ERROR: Could not find a version that satisfies the requirement tflite-runtime==2.7.0.post1 (from pycoral) (from versions: none)
ERROR: No matching distribution found for tflite-runtime==2.7.0.post1

I wonder if the fact the assets are split into pycoral and tflite_runtime wheels is relevant? Of note, I don't know how this system works, and was previously confused about the difference between the two libs.

I guess they split them originally so that tflite-runtime could be used for other things then pycoral.

Regarding your error, you need to install tflite-runtime before pycoral..

[David-OConnor]

I appreciate the info; I have it installed.

[oberluz]

@David-OConnor @baudneo

Pycoral for Tensorflow 2.13.0 has been released and includes wheels for python 3.11: https://github.com/oberluz/pycoral/releases

Tensorflow Python 3.11 support was introduced in v2.12.0 : https://github.com/tensorflow/tensorflow/releases/tag/v2.12.0 so it is not possible to create pycoral wheels for any release previous to v2.12.0

I'll be making a release for 2.12.0 and 2.14.0 soon.

If you build it on any arch other then amd64 (k8) or aarch64 please let me know and I'll add them to the releases.

Cheers

[oberluz]

Just a as a final followup. All of the google projects I've had to work on to get this going (pycoral, libcoral and libedgetpu) seem to have been stopped for quite a while. I don't hold any hope they may be retaken so from now on if you need anything please use the discussions page in my repo: https://github.com/oberluz/pycoral/discussions to make further requests or anything else. I won't be following this discussion here any longer. Cheers

[David-OConnor]

Great! Do you know how to solve the issue, when installing from the wheel: ERROR: Could not find a version that satisfies the requirement tflite-runtime==2.13.0 ? Ty. I have tensorflow==2.13.0 installed. Of note, this is similar to the error I received when attempting to build the wheels myself.

edit: It appears that tflite-runtime doesn't support 2.13.0 on Py3.11. Open to ideas to workaround this. I'm curious how you got it to work on your system. I think I'll hold off until your 2.14 release, since this error should hopefully go away then.

[oberluz]

Raise an issue in my repo and state the exact wheels you are trying to install

…

On Sun, 5 Nov 2023 at 16:19, David-OConnor ***@***.***> wrote: Great! Do you know how to solve the issue, when installing from the wheel: ERROR: Could not find a version that satisfies the requirement tflite-runtime==2.13.0 ? Ty. I have tensofrlow==2.13.0 installed. Of note, this is similar to the error I received when attempting to build the wheels myself. — Reply to this email directly, view it on GitHub <#85 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKFR7ST63PDX53QG7VT34TYC6N7NAVCNFSM56DDBEI2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZZGM3TMNRTGE3Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[oberluz]

apologies, I hadn't enabled the issues tab, try now...

…

On Sun, 5 Nov 2023 at 16:29, Paul Nader ***@***.***> wrote: Raise an issue in my repo and state the exact wheels you are trying to install On Sun, 5 Nov 2023 at 16:19, David-OConnor ***@***.***> wrote: > Great! Do you know how to solve the issue, when installing from the > wheel: ERROR: Could not find a version that satisfies the requirement > tflite-runtime==2.13.0 ? Ty. I have tensofrlow==2.13.0 installed. Of > note, this is similar to the error I received when attempting to build the > wheels myself. > > — > Reply to this email directly, view it on GitHub > <#85 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAKFR7ST63PDX53QG7VT34TYC6N7NAVCNFSM56DDBEI2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZZGM3TMNRTGE3Q> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

[nmaas87]

Any idea on how to build this on windows?
It seems like both the Docker and Bazle way seem not to work at all with 3.10 nor 3.11 using Oberluz Tags/Repo - but it seems to be an initial problem with Googles Windows scripts.

[oberluz]

Hi, As I mentioned in my last post please raise an issue in my repo. Explain in detail your environment and what you ae doing and I might be able to help you. I won't be answering questions from this thread any longer...

…

On Sun, 5 Nov 2023 at 19:41, Nico Maas ***@***.***> wrote: Any idea on how to build this on windows? It seems like both the Docker and Bazle way seem not to work at all with 3.10 nor 3.11 using Oberluz Tags/Repo - but it seems to be an initial problem with Googles Windows scripts. — Reply to this email directly, view it on GitHub <#85 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKFR7WPDOFDD52I7VVLZCDYC7MUDAVCNFSM56DDBEI2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZZGM4DCMZVGUYQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[nmaas87]

To all the people interested in the windows builds, I am talking this over all oberluz repo: oberluz#3 - looks like it does not work for me, but if you have any inputs or ideas you can give them there.

[arne182]

I have a new raspberry Pi 5 and it comes with 3.11 is there any update on this?

[baudneo]

I have a new raspberry Pi 5 and it comes with 3.11 is there any update on this?

See the work that @oberluz has done on this issue.

[vinzbarbuto]

Hello everyone, I'm reaching out for an update regarding the official wheels. Currently, I urgently require Python 3.10 to integrate with an external framework. However, I'm encountering a roadblock with pycoral on my Dev Board, which operates on Python 3.7. To address this, I installed Python 3.10, but unfortunately, I'm facing difficulties in downloading the pycoral API as it seems to be unsupported by Python 3.10 at the moment.

I attempted to install pycoral using the provided link for the wheels. Despite successful installation, I encountered the following error message: ImportError: /lib/aarch64-linux-gnu/libc.so.6: version GLIBC_2.32' not found. Upon investigating further by executing ldd --version on my machine, it indicates ldd (Debian GLIBC 2.28-10) 2.28.

Does anyone have any insights or suggestions on how to overcome these challenges?

[adelmiljkovic]

Does anyone have any insights or suggestions on how to overcome these challenges?

Releases for Python 3.10 can be found here but I didn't test
https://github.com/oberluz/pycoral/releases

[baudneo]

Hello everyone, I'm reaching out for an update regarding the official wheels. Currently, I urgently require Python 3.10 to integrate with an external framework. However, I'm encountering a roadblock with pycoral on my Dev Board, which operates on Python 3.7. To address this, I installed Python 3.10, but unfortunately, I'm facing difficulties in downloading the pycoral API as it seems to be unsupported by Python 3.10 at the moment.

I attempted to install pycoral using the provided link for the wheels. Despite successful installation, I encountered the following error message: ImportError: /lib/aarch64-linux-gnu/libc.so.6: version GLIBC_2.32' not found. Upon investigating further by executing ldd --version on my machine, it indicates ldd (Debian GLIBC 2.28-10) 2.28.

Does anyone have any insights or suggestions on how to overcome these challenges?

You need to build it yourself on your machine, thats the only reliable way to make sure it will work for your system. See the head of this issue for building instructions and some workarounds needed to build.