feat: Python 3.10 support #5693

ickk · 2021-04-02T13:26:58Z

Python 3.10 is coming. The new match,case stuff is particularly exciting to a developer coming from Rust or Haskell.

PyInstaller is a great tool for the developer when she wants to distribute a software application to the end-user without requiring the end-user to have any particular version of python installed. This is can simplify the developer's maintenance efforts - rather than trying to keep her software compatible across multiple versions of Python - and can allow her to take advantage of newer language features, like those found in the newest version of Python.

It would be a great addition for PyInstaller to add support for Python 3.10.

What I have tried:
I have tried, naively, to run PyInstaller against python 3.10.0a6 but immediately ran into at least one issue:

_bootlocale has been removed in cPython 3.10 and has been replaced with a new _locale._get_locale_encoding()
excluding _bootlocale from PY3_BASE_MODULES for python 3.10 in PyInstaller/compat.py allows PyInstaller to generate an executable, however from here I kept running into issues when running the resulting binary.

It is not obvious to me what needs to be done to support python 3.10, That said, if desirable I am perfectly happy to do what I can with a little direction.

The text was updated successfully, but these errors were encountered:

bwoodsend · 2021-04-02T14:49:47Z

I can get a functional executable on Linux if I:

Replace _bootlocale with _locale as you said in compat.PY3_BASE_MODULES
Add struct to compat.PY3_BASE_MODULES
Copy/paste the importer/loader runtime scripts from PyInstaller/loader straight into the built application's root.

Number 3 is a hack - we'll need to work out why the original approach to inject those runtime modules isn't working.

rokm · 2021-04-02T15:11:19Z

Number 3 is a hack - we'll need to work out why the original approach to inject those runtime modules isn't working.

Because PI_PyObject_CallFunction calls are erroring out with PY_SSIZE_T_CLEAN macro must be defined for '#' formats. It might be enough to have our PI_PyObject_CallFunction point to _PyObject_CallFunction_SizeT instead of PyObject_CallFunction for Python 3.10...

rokm · 2021-04-02T15:23:41Z

We'll need to address pyvers field in COOKIE becoming 40 under python 3.10... for example, encoding it as

pyvers = sys.version_info[0] * 100 + sys.version_info[1]

instead of

pyvers = sys.version_info[0] * 10 + sys.version_info[1]

(it's a 32-bit int, so there's no shortage of space...).

SLBIAM · 2021-07-22T17:03:44Z

The issue is NOT resolved fully.

I have Beta 4 of Python 10 installed on my (Windows 10) PC (which I must use because I really need pattern matching feature only available in v3.10):

C:\Users\myname\Documents\Projects\Project Migration\SeleniumExamplePy>py -3 --version
Python 3.10.0b4

and I made sure my pyinstaller is up to date:

    C:\Users\myname\Documents\Projects\Project Migration\SeleniumExamplePy>pip install --upgrade pyinstaller
Requirement already satisfied: pyinstaller in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (4.4)
Requirement already satisfied: pefile>=2017.8.1 in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pyinstaller) (2021.5.24)
Requirement already satisfied: pyinstaller-hooks-contrib>=2020.6 in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pyinstaller) (2021.2)
Requirement already satisfied: pywin32-ctypes>=0.2.0 in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pyinstaller) (0.2.0)
Requirement already satisfied: setuptools in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pyinstaller) (56.0.0)
Requirement already satisfied: altgraph in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pyinstaller) (0.17)
Requirement already satisfied: future in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pefile>=2017.8.1->pyinstaller) (0.18.2)

I even tried installing the latest dev build:

pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip
Collecting https://github.com/pyinstaller/pyinstaller/archive/develop.zip
  Downloading https://github.com/pyinstaller/pyinstaller/archive/develop.zip
     / 4.0 MB 2.2 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: pyinstaller-hooks-contrib>=2020.6 in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pyinstaller==5.0.dev0) (2021.2)
Requirement already satisfied: pefile>=2017.8.1 in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pyinstaller==5.0.dev0) (2021.5.24)
Requirement already satisfied: altgraph in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pyinstaller==5.0.dev0) (0.17)
Requirement already satisfied: pywin32-ctypes>=0.2.0 in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pyinstaller==5.0.dev0) (0.2.0)
Requirement already satisfied: setuptools in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pyinstaller==5.0.dev0) (56.0.0)
Requirement already satisfied: future in c:\users\myname\appdata\local\programs\python\python310\lib\site-packages (from pefile>=2017.8.1->pyinstaller==5.0.dev0) (0.18.2)
Building wheels for collected packages: pyinstaller
  Building wheel for pyinstaller (PEP 517) ... done
  Created wheel for pyinstaller: filename=pyinstaller-5.0.dev0-py3-none-any.whl size=2670672 sha256=d7b490f711282e8b635300a5eb2ebf47d78b7aa21dacc518fdf1ee8606a4aac0
  Stored in directory: C:\Users\myname\AppData\Local\Temp\pip-ephem-wheel-cache-rseun6wp\wheels\d6\02\6d\a387c4f56a9895d41bc6868742c7f6ad092c79ffce4b680952
Successfully built pyinstaller
Installing collected packages: pyinstaller
  Attempting uninstall: pyinstaller
    Found existing installation: pyinstaller 4.4
    Uninstalling pyinstaller-4.4:
      Successfully uninstalled pyinstaller-4.4
Successfully installed pyinstaller-5.0.dev0

I am still getting ImportError: No module named _bootlocale error while trying to convert my python code (that I thoroughly tested) to executable:

C:\Users\myname\Documents\Projects\Project Migration\SeleniumExamplePy>pyinstaller --onefile --clean ExtractRules.spec
83 INFO: PyInstaller: 4.4
83 INFO: Python: 3.10.0b4
100 INFO: Platform: Windows-10-10.0.18363-SP0
110 INFO: UPX is not available.
110 INFO: Removing temporary files and cleaning cache in C:\Users\myname\AppData\Local\pyinstaller
113 INFO: Extending PYTHONPATH with paths
['C:\\Users\\myname\\Documents\\Projects\\Project '
 'Migration\\SeleniumExamplePy',
 'C:\\Users\\myname\\Documents\\Projects\\Project '
 'Migration\\SeleniumExamplePy']
126 INFO: checking Analysis
126 INFO: Building Analysis because Analysis-00.toc is non existent
126 INFO: Initializing module dependency graph...
126 INFO: Caching module graph hooks...
139 INFO: Analyzing base_library.zip ...
3449 INFO: Processing pre-find module path hook distutils from 'c:\\users\\myname\\appdata\\local\\programs\\python\\python310\\lib\\site-packages\\PyInstaller\\hooks\\pre_find_module_path\\hook-distutils.py'.
3449 INFO: distutils: retargeting to non-venv dir 'c:\\users\\myname\\appdata\\local\\programs\\python\\python310\\lib'
Traceback (most recent call last):
  File "c:\users\myname\appdata\local\programs\python\python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\myname\appdata\local\programs\python\python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\myname\AppData\Local\Programs\Python\Python310\Scripts\pyinstaller.exe\__main__.py", line 7, in <module>
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\__main__.py", line 126, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\__main__.py", line 65, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\building\build_main.py", line 758, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\building\build_main.py", line 705, in build
    exec(code, spec_namespace)
  File "ExtractRules.spec", line 7, in <module>
    a = Analysis(['ExtractRules.py'],
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\building\build_main.py", line 254, in __init__
    self.__postinit__()
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\building\datastruct.py", line 159, in __postinit__
    self.assemble()
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\building\build_main.py", line 354, in assemble
    self.graph = initialize_modgraph(
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\depend\analysis.py", line 882, in initialize_modgraph
    graph = PyiModuleGraph(
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\depend\analysis.py", line 122, in __init__
    self._analyze_base_modules()
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\depend\analysis.py", line 289, in _analyze_base_modules
    self._base_modules = [mod
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\depend\analysis.py", line 291, in <listcomp>
    for mod in self.import_hook(req)]
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\lib\modulegraph\modulegraph.py", line 1505, in import_hook
    target_package, target_module_partname = self._find_head_package(
  File "c:\users\myname\appdata\local\programs\python\python310\lib\site-packages\PyInstaller\lib\modulegraph\modulegraph.py", line 1711, in _find_head_package
    raise ImportError("No module named " + target_package_name)
ImportError: No module named _bootlocale

I was able to resolve the issue and build the executable by following only the first step of the bwoodsend's comment above
"Replace _bootlocale with _locale as you said in compat.PY3_BASE_MODULES"
I did not need to do steps 2 and 3, but wanted to let you know that the problem is still not fully resolved without editing compat.py module file manually.

rokm · 2021-07-22T17:09:17Z

The issue is NOT resolved fully.

Do note that the linked PR has not been merged yet nor has this issue been marked as resolved... and it likely won't be until the stable 3.10 release and the python packages that our CI tests against release compatible wheels.

bwoodsend · 2021-07-22T17:10:49Z

Use:

pip install https://github.com/rokm/pyinstaller/archive/refs/heads/python-3.10.zip

to get the 3.10 support branch.

rokm · 2021-07-22T17:13:45Z

Hmm, think I'll rebase and push it again, to see how the tests turn out this time. I see that failing test with 3.10 on Linux and macOS was that recursion test that we removed the other day...

rokm · 2021-07-22T17:14:38Z

Use:

pip install https://github.com/rokm/pyinstaller/archive/refs/heads/python-3.10.zip

to get the 3.10 support branch.

This branch requires bootloader to be rebuilt (due to pyvers change).

bwoodsend · 2021-07-22T17:18:50Z

In that case you'll have to do a proper git clone, build the bootloaders, then locally install. Don't use the oneliner because it won't work.

SLBIAM · 2021-07-22T23:14:14Z

I appreciate quick responses folks. Given that simple small change that I described resolved an immediate issue for me, in the interest of saving time I will refrain from going the whole distance for now ("do a proper git clone, build the bootloaders, then locally install"). I just wanted to bring the issue to your attention to give back to the community. I did not realize that linked PR has not been merged yet--hopefully that will resolve the issue permanently when done.

eric15342335 · 2021-09-03T13:17:53Z

do we have a timeline for this?

bwoodsend · 2021-09-03T15:53:00Z

Not really. We'll merge the pull request once 3.10 is officially released and try to do a v4.6 release as soon as possible afterwards. I'll give the PR another rebase now to see if anything new has changed.

jay0lee · 2021-10-04T22:45:34Z

just pinging this now that 3.10.0 is official:

https://pythoninsider.blogspot.com/2021/10/python-3100-is-available.html

michaelkozakovsc · 2021-10-05T06:09:47Z

3.10 is also the default release on Travis CI on windows, which results in broken builds that involve pyinstaller

anibal2j · 2021-10-05T18:59:19Z

Also looking for an update in regards to this. I just installed 3.10 on a Windows 10 workstation and need to get pyinstaller to work but can't. Any ETA will be greatly appreciated.

rokm · 2021-10-05T19:05:49Z

Also looking for an update in regards to this. I just installed 3.10 on a Windows 10 workstation and need to get pyinstaller to work but can't. Any ETA will be greatly appreciated.

You can always use #5694 (v.5/develop branch) or #6274 (v.4 branch) if you absolutely need to use python 3.10 a day after it was released... (but you need to do an actual git checkout, rebuild the bootloader, and then install from the modified checkout).

zetaloop · 2021-10-06T02:24:38Z

I tried --exclude-module _bootlocale and it worked for me properly

anibal2j · 2021-10-14T02:01:11Z

I tried as well with the --exclude-module _bootlocale and it worked with a simple Hello world program. However, with a slightly more complex program I'm now getting this:

Traceback (most recent call last):
  File "main.py", line 7, in <module>
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
  File "requests\__init__.py", line 133, in <module>
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
  File "requests\utils.py", line 41, in <module>
  File "certifi\core.py", line 37, in where
  File "contextlib.py", line 135, in __enter__
  File "importlib\_common.py", line 87, in _tempfile
  File "tempfile.py", line 337, in mkstemp
  File "tempfile.py", line 249, in _mkstemp_inner
TypeError: can only concatenate str (not "method") to str
[5848] Failed to execute script 'main' due to unhandled exception!

I'm using these Python libraries:

from prettytable import PrettyTable
import requests
from tqdm import tqdm

Could that be the reason? Any way to get around this?

rokm · 2021-10-14T08:49:48Z

Could that be the reason? Any way to get around this?

Packages that use importlib.resources (in your case, requests) are incompatible with PyInstaller under python 3.10 unless a custom PyInstaller-compatible importlib.resources reader is implemented. So you need to use a branch with proper 3.10 support (or back-port the importlib.resources PR to your 4.5.1 installation).

ickk · 2021-10-15T04:02:07Z

🎉

Davido17 · 2021-10-28T14:21:12Z

i

I tried --exclude-module _bootlocale and it worked for me properly

I used that as well, it made the exe file but the app is not working. it just pops up n goes on its own

Legorooj · 2021-10-29T12:17:36Z

I've just release PyInstaller 4.6 onto PyPI. This release includes support for Python 3.10.

ickk added the feature Feature request label Apr 2, 2021

bwoodsend added the bug label Apr 2, 2021

rokm mentioned this issue Apr 2, 2021

Add support for Python 3.10 #5694

Merged

machawk1 mentioned this issue Oct 6, 2021

Update to build with Python 3.10 (dependent on wxPython support) machawk1/wail#543

Closed

1 task

branchvincent mentioned this issue Oct 6, 2021

pyinstaller: migrate to python@3.10 Homebrew/homebrew-core#86640

Closed

Avnsx mentioned this issue Oct 9, 2021

Incompatible with Python 3.10 brentvollebregt/auto-py-to-exe#201

Closed

bwoodsend closed this as completed in #5694 Oct 14, 2021

The-Compiler mentioned this issue Nov 1, 2021

PyInstaller 4.6 qutebrowser/qutebrowser#6792

Closed

4 tasks

madtatu-development mentioned this issue Nov 2, 2021

Incompatible with python 3.10.0 (UPDATED) brentvollebregt/auto-py-to-exe#205

Closed

fangxx3863 mentioned this issue Dec 28, 2021

macOS support (experimental) Taiko2k/Tauon#577

Open

Fizzadar mentioned this issue Mar 31, 2022

Upgrade flask to >=2.0 to support python 3.10 and further release Oxygem/Kanmail#201

Open

Hurgotron mentioned this issue Oct 13, 2022

unknown encoding: ascii in chroot jrosdahl/miniircd#51

Closed

github-actions bot locked as resolved and limited conversation to collaborators Nov 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Python 3.10 support #5693

feat: Python 3.10 support #5693

ickk commented Apr 2, 2021

bwoodsend commented Apr 2, 2021

rokm commented Apr 2, 2021

rokm commented Apr 2, 2021

SLBIAM commented Jul 22, 2021

rokm commented Jul 22, 2021

bwoodsend commented Jul 22, 2021

rokm commented Jul 22, 2021

rokm commented Jul 22, 2021 •

edited

Loading

bwoodsend commented Jul 22, 2021

SLBIAM commented Jul 22, 2021

eric15342335 commented Sep 3, 2021

bwoodsend commented Sep 3, 2021

jay0lee commented Oct 4, 2021

michaelkozakovsc commented Oct 5, 2021

anibal2j commented Oct 5, 2021

rokm commented Oct 5, 2021 •

edited

Loading

zetaloop commented Oct 6, 2021

anibal2j commented Oct 14, 2021

rokm commented Oct 14, 2021 •

edited

Loading

ickk commented Oct 15, 2021

Davido17 commented Oct 28, 2021

Legorooj commented Oct 29, 2021

feat: Python 3.10 support #5693

feat: Python 3.10 support #5693

Comments

ickk commented Apr 2, 2021

bwoodsend commented Apr 2, 2021

rokm commented Apr 2, 2021

rokm commented Apr 2, 2021

SLBIAM commented Jul 22, 2021

rokm commented Jul 22, 2021

bwoodsend commented Jul 22, 2021

rokm commented Jul 22, 2021

rokm commented Jul 22, 2021 • edited Loading

bwoodsend commented Jul 22, 2021

SLBIAM commented Jul 22, 2021

eric15342335 commented Sep 3, 2021

bwoodsend commented Sep 3, 2021

jay0lee commented Oct 4, 2021

michaelkozakovsc commented Oct 5, 2021

anibal2j commented Oct 5, 2021

rokm commented Oct 5, 2021 • edited Loading

zetaloop commented Oct 6, 2021

anibal2j commented Oct 14, 2021

rokm commented Oct 14, 2021 • edited Loading

ickk commented Oct 15, 2021

Davido17 commented Oct 28, 2021

Legorooj commented Oct 29, 2021

rokm commented Jul 22, 2021 •

edited

Loading

rokm commented Oct 5, 2021 •

edited

Loading

rokm commented Oct 14, 2021 •

edited

Loading