Support for python 3.10

[sshankar619]

pip install fails for python 3.10
Is python 3.10 supported?

why 3.10 is not supported in docker file

pynini/third_party/Dockerfile

Line 35 in 0696c50

RUN rm -rd /opt/python/cp310-cp310

Can wheel file be provided for python 3.10

[kylebgorman]

I'll test it on next release. It wasn't working out of the box and I didn't feel like debugging at the time.

You can always just use Conda instead: it supports Python 3.7-3.10 on MacOS (x86 and the new ARM family), Windows, and Linux (x86).

In contrast, the Dockerfile just generates an alternative wheel for Linux x86_64.

Finally, you can edit and run the Dockerfile yourself if you absolutely need Python 3.10 + Linux + x86_64 support but can't use Conda.

[gayu-thri]

Do we have any update on this?

I have got a similar requirement of higher Python version support for 3.11.

For this case, I understand we can use Conda, but if it could be provided as a pip package for Python 3.11, that would be great as one of the projects I'm working on requires Pynini and it is a poetry-based project with PyPI configured as the default package source, and the existing dependencies are also mostly pip packages.​

[kylebgorman]

I’ll include this in on the next release, but making a release may take me as much as a few engineer-days, and usually requires me to update the rest of the stack too. I can’t promise I’ll get to this in the next few weeks, I’m sorry, but I likely will by sometime in April.I don’t know if it will work but the Dockerfile I use to generate the manylinux wheels is checked in to this repository. You could try to modify it to support 3.10 and 3.11, and if you report that it works with minimal modification I can put the wheels for 3.10-11 on PyPI without upgrading Pynini itself. No pressure though.On Mar 16, 2023, at 5:13 AM, Gayathri Ethiraj ***@***.***> wrote: Do we have any update on this? I have got a similar requirement of higher Python version support for 3.11. For this case, I understand we can use Conda, but if it could be provided as a pip package for Python 3.11, that would be great as one of the projects I'm working on requires Pynini and it is a poetry-based project with PyPI configured as the default package source, and the existing dependencies are also mostly pip packages.​ —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

[gayu-thri]

I tried building wheels using the Dockerfile by commenting the below line.

pynini/third_party/Dockerfile

Line 35 in 0696c50

RUN rm -rd /opt/python/cp310-cp310

• For pynini 2.1.5, the tests seems to be running fine in both python 3.10 and 3.11.
• For pynini 2.1.4, it looks like some tests are failing with some errors like failing to compile a string as shared in the image. Tests failed mostly under Python 3.10.10 and 3.11.2 versions.

However, when I try to install the respective generated cp wheel file (Ex: pynini-2.1.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl) for Python 3.10 and 3.11 (for both 2.1.5 and 2.1.4 pynini) in an environment and test working of pynini, it's working fine. The latest version of pynini in pypi also has only the cp wheel file for Python 3.9 and it's working.

If cp wheel files are usable, would it be okay for us to have this support for 3.10 and 3.11 as well? Please let me know otherwise if I am missing something here.

[kylebgorman]

Okay, I will try to generate the manylinux wheel for Pynini 2.1.5 on Python 3.10 and Python 3.11. May need some time to get it done.

I am not really interested in maintaining Pynini 2.1.4, which is quite out of date now.

I think you may be misreading the test output there: at the bottom it says all 146 tests passed. Some tests deliberately raise error messages and catch the resulting exceptions, but that doesn't mean the test failed...

[gayu-thri]

I think you may be misreading the test output there: at the bottom it says all 146 tests passed. Some tests deliberately raise error messages and catch the resulting exceptions, but that doesn't mean the test failed...

Thanks for clarifying. If I understand correctly, those tests are some kind of negative testing and hence raising errors.

Okay, I will try to generate the manylinux wheel for Pynini 2.1.5 on Python 3.10 and Python 3.11. May need some time to get it done.

Sure. That shouldn't be a problem, please try it at your earliest convenience.

[kylebgorman]

I just uploaded the cp310 (CPython 3.10) x86_64 manylinux wheel for Pynini 2.1.5. I'll support Python 3.10 on the next release too. The dockerfile setup we have now doesn't seem to pull in Python 3.11 so that may be some additional work in the future.

[gayu-thri]

The dockerfile setup we have now doesn't seem to pull in Python 3.11 so that may be some additional work in the future.

Just wanted to clarify on this point.

When I was trying to generate wheel files using the Dockerfile, I was able to get Python 3.11 pulled (cp311) and the above message that I shared earlier stating that it's working fine in Python 3.11 also was by making use of this wheel file.

[kylebgorman]

I don’t think I saw Python 3.11 wheels on my system but it’s possible I missed something. Is it likely we got different outputs from the same dockerfile?

…

On Wed, Apr 5, 2023 at 1:34 AM Gayathri Ethiraj ***@***.***> wrote: Just wanted to follow up on this, it would be helpful if you could provide me with clarification on this matter @kylebgorman <https://github.com/kylebgorman> The dockerfile setup we have now doesn't seem to pull in Python 3.11 so that may be some additional work in the future. Just wanted to clarify on this point. When I was trying to generate wheel files using the Dockerfile, I was able to get Python 3.11 pulled (cp311) and the above message that I shared earlier stating that it's working fine in Python 3.11 also was by making use of this wheel file. [image: image] <https://user-images.githubusercontent.com/39553592/229047449-674b7603-1d90-4757-8a8a-23622c850f9e.png> — Reply to this email directly, view it on GitHub <#64 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABG4OOKO7QZUBO7T5LNPLLW7UOCBANCNFSM6AAAAAAUSU45UQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[gayu-thri]

I have opened a new issue #65 and replied there (#65 (comment)), we could have the further discussion there as this issue is already closed and is pertaining to Python 3.10 support.