Binary built with PyInstaller doesn't recognize Python source file encoding in case the file doesn't have .py extension

[skif443]

Binary built with PyInstaller doesn't recognize Python source file encoding in case the file doesn't have .py extension.

Context information (for bug reports)

• Output of pyinstaller --version: 6.0.0
• Version of Python: Python 3.9.2
• Platform: Linux (Astra Linux 1.7.4)
• How you installed Python: pyenv
• Did you also try this on another platform? Does it work there? - Didn't try

PyInstaller command:

pyinstaller --clean main.spec

Spec file:

# -*- mode: python ; coding: utf-8 -*-

import os
import ansible


a = Analysis(
    ['aa-setup'],
    pathex=[],
    binaries=[],
    datas=[
        (os.path.dirname(ansible.__file__), 'ansible')
    ],
    hiddenimports=['ansible'],
    hookspath=['pyinstaller-hooks'],
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    noarchive=False,
)
pyz = PYZ(a.pure)

exe = EXE(
    pyz,
    a.scripts,
    a.binaries,
    a.datas,
    [],
    name='aa-setup',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    upx_exclude=[],
    runtime_tmpdir=None,
    console=True,
    disable_windowed_traceback=False,
    argv_emulation=False,
    target_arch=None,
    codesign_identity=None,
    entitlements_file=None,
)

A minimal example program which shows the error

!!! Make sure that file name with the following code doesn't contain any extension (.py for example)

#!./.venv/bin/python3 -W ignore::DeprecationWarning
import traceback

def test():
    raise Exception

def main():
    try:
        test()
    except Exception:
        traceback.print_exc()


if __name__ == '__main__':
    main()

Stacktrace / full error message

Traceback (most recent call last):
  File "aa-setup", line 9, in main
  File "aa-setup", line 5, in test
Exception

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tokenize.py", line 330, in find_cookie
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf8 in position 40: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "aa-setup", line 15, in <module>
  File "aa-setup", line 11, in main
  File "traceback.py", line 163, in print_exc
  File "traceback.py", line 103, in print_exception
  File "traceback.py", line 508, in __init__
  File "traceback.py", line 366, in extract
  File "traceback.py", line 288, in line
  File "linecache.py", line 30, in getline
  File "linecache.py", line 46, in getlines
  File "linecache.py", line 136, in updatecache
  File "tokenize.py", line 394, in open
  File "tokenize.py", line 371, in detect_encoding
  File "tokenize.py", line 335, in find_cookie
SyntaxError: invalid or missing encoding declaration for 'aa-setup'
[1935] Failed to execute script 'aa-setup' due to unhandled exception!

Please also see https://github.com/pyinstaller/pyinstaller/wiki/How-to-Report-Bugs
for more about what would use to solve the issue.

[rokm]

a = Analysis(
   ['main.py'],

Traceback (most recent call last):
  File "aa-setup", line 50, in <module>
  File "aa-setup", line 46, in main
  File "traceback.py", line 163, in print_exc

Is the entry point called main.py as indicated in the spec file, or is it called aa-setup as indicated in the traceback?

Can we see the whole entry-point script? There seem to be at least 40 more lines to it, and one of it contains an non-ASCII character that likely triggers the issue...

[skif443]

My bad, made a mistake in bug description.

Here's correct spec part

a = Analysis(
   ['main'],

[rokm]

If the entry point for frozen application is main, what is aa-setup then? An external script?

[skif443]

Updated Bug description one more time. Here's code for reproducing bug:

#!./.venv/bin/python3 -W ignore::DeprecationWarning
import traceback

def test():
    raise Exception

def main():
    try:
        test()
    except Exception:
        traceback.print_exc()


if __name__ == '__main__':
    main()

[rokm]

And if you freeze this script (presumably this is main) using PyInstaller, what is the traceback?

[skif443]

Sorry, updated Bug description once again. aa-setup is a correct file name that contains code mentioned in bug description. Please forget about main file, I've removed its mentions from bug description.

[rokm]

Does the error occur only if you run the program from dist/aa-setup directory (./aa-setup), but not if you run if from elsewhere (e.g., from dist directory, ./aa-setup/aa-setup)?

[skif443]

I do copy aa-setup to another directory (not dist/aa-setup) and then execute it. In this case issue is reproducible.

[rokm]

Okay, but it is reproducible only if the executable is in the current working directory, right? I think that happens because traceback and linecache try to read the source code based on the file name of the entry-point script, which in this case coincides with the executable name.

I think as a work-around, if entry-point script name has no suffix, we will add the .py to the name when compiling it. It is probably the least invasive way to take care of this...