Bus error in latest PyInstaller 5.10.0 on Raspberry Pi 32bit Buster OS

[gaamaaresosa]

System Used:
Raspberry Pi 4 with Buster Rasbian OS 32bit
PyInstaller 5.10.0

While running the built binary I get below error and application exits.
Bus error

Since 4 months I use Pyinstaller-5.9.0 10 to 15 times a day on my project
And no issue faced.
Yesterday (13Apr2023) I have updated PyInstaller 5.10.0
After that I get this error.
Bus error

Now I again downgraded to Pyinstaller-5.9.0
Everything working fine.

Please check this bug.

[bwoodsend]

Please check this bug.

We would we had any idea how to reproduce this error. Can you please fill out the issue template.

[gaamaaresosa]

Please check this bug.

We would we had any idea how to reproduce this error. Can you please fill out the issue template.

This is what I use to build:
sudo pyinstaller -c -F --key 'MyKey' --clean --upx-dir=/usr/bin/ /home/pi/MyApp.py

If you need sample MyApp.py then I need to write , test and send
I think if you test any simple python3 code with Raspberry Pi 4 32bit Buster OS, you could see the issue

[rokm]

The bus error happens with any python script you try to freeze? E.g., also with just a print("Hello world!") one?

[gaamaaresosa]

The bus error happens with any python script you try to freeze? E.g., also with just a print("Hello world!") one?

I have 2 binaries in my current project.
Both are same issue with latest PyInstaller 5.10.

Do you need me to make a one line python code and reproduct the issue in latest PyInstaller 5.10 ?
Ok, I will do now please give me few hours.

[rokm]

Actually, what I'd like you to do is to try installing PyInstaller from source (assuming your current installation was just using pip install --upgrade pyinstaller):

pip uninstall pyinstaller # Or create a new clean venv
pip install git+https://github.com/pyinstaller/pyinstaller.git

This should trigger local bootloader compilation (for which you need to have gcc and zlib1g-dev installed via apt). If you install PyInstaller this way and rebuild your application (or one-line test script), do you still get the bus error?

[gaamaaresosa]

pip uninstall pyinstaller

I got it what you say.
First I will reproduce the issue with my one line test code and confirm the issue.
Then I will your steps.

[gaamaaresosa]

Here I have reproduced with oneline code:
This test with just (Not fresh install)
sudo pip3 install --upgrade pyinstaller==5.10

NOTE:
I am not convinced doing fresh install, because we always do updates right ?
But I will do as you have suggested.

BusErrorTest.py

#!/usr/bin/python3
print('This is a PyInsyaller 5.10 Bus error test')

While Building:

$ sudo pyinstaller -c -F --key 'MySecuredKey' --clean --upx-dir=/usr/bin/ /home/pi/PyInstallerTest/BusErrorTest.py
414 DEPRECATION: Bytecode encryption will be removed in PyInstaller v6. Please remove your --key=xxx argument to avoid breakages on upgrade. For the rationale/alternatives see #6999
414 INFO: PyInstaller: 5.10.0
414 INFO: Python: 3.7.3
418 INFO: Platform: Linux-6.1.23-v8+-aarch64-with-debian-10.13
420 INFO: wrote /home/pi/PyInstallerTest/BusErrorTest.spec
428 INFO: UPX is available.
430 INFO: Removing temporary files and cleaning cache in /root/.cache/pyinstaller
442 DEPRECATION: Bytecode encryption will be removed in PyInstaller v6. Please remove cipher and block_cipher parameters from your spec file to avoid breakages on upgrade. For the rationale/alternatives see #6999
443 INFO: Extending PYTHONPATH with paths
['/home/pi/PyInstallerTest']
1087 INFO: Will encrypt Python bytecode with provided cipher key
1088 INFO: checking Analysis
1089 INFO: Building Analysis because Analysis-00.toc is non existent
1089 INFO: Initializing module dependency graph...
1097 INFO: Caching module graph hooks...
1134 INFO: Analyzing base_library.zip ...
3759 INFO: Loading module hook 'hook-encodings.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
5334 INFO: Loading module hook 'hook-pickle.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
5618 INFO: Loading module hook 'hook-heapq.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
9295 INFO: Caching module dependency graph...
9676 INFO: running Analysis Analysis-00.toc
9775 INFO: Analyzing /home/pi/PyInstallerTest/BusErrorTest.py
9779 INFO: Analyzing hidden import 'tinyaes'
9780 INFO: Processing module hooks...
9816 INFO: Looking for ctypes DLLs
9816 INFO: Analyzing run-time hooks ...
9831 INFO: Looking for dynamic libraries
10922 INFO: Looking for eggs
10922 INFO: Python library not among binary dependencies. Performing additional search...
11118 INFO: Using Python library /lib/arm-linux-gnueabihf/libpython3.7m.so.1.0
11124 INFO: Warnings written to /home/pi/PyInstallerTest/build/BusErrorTest/warn-BusErrorTest.txt
11169 INFO: Graph cross-reference written to /home/pi/PyInstallerTest/build/BusErrorTest/xref-BusErrorTest.html
11194 INFO: checking PYZ
11194 INFO: Building PYZ because PYZ-00.toc is non existent
11195 INFO: Building PYZ (ZlibArchive) /home/pi/PyInstallerTest/build/BusErrorTest/PYZ-00.pyz
11760 INFO: Building PYZ (ZlibArchive) /home/pi/PyInstallerTest/build/BusErrorTest/PYZ-00.pyz completed successfully.
11765 INFO: checking PKG
11765 INFO: Building PKG because PKG-00.toc is non existent
11765 INFO: Building PKG (CArchive) BusErrorTest.pkg
15855 INFO: Building PKG (CArchive) BusErrorTest.pkg completed successfully.
15859 INFO: Bootloader /usr/local/lib/python3.7/dist-packages/PyInstaller/bootloader/Linux-32bit-arm/run
15859 INFO: checking EXE
15859 INFO: Building EXE because EXE-00.toc is non existent
15859 INFO: Building EXE from EXE-00.toc
15860 INFO: Copying bootloader EXE to /home/pi/PyInstallerTest/dist/BusErrorTest
15860 INFO: Appending PKG archive to custom ELF section in EXE
15900 INFO: Building EXE from EXE-00.toc completed successfully.

Testing:

$ sudo ./dist/BusErrorTest
Bus error

[gaamaaresosa]

Here is a clean install and same issue:
(As you had suggessted)

$ sudo pip3 uninstall pyinstaller

Found existing installation: pyinstaller 5.10.0
Uninstalling pyinstaller-5.10.0:
Would remove:
/usr/local/bin/pyi-archive_viewer
/usr/local/bin/pyi-bindepend
/usr/local/bin/pyi-grab_version
/usr/local/bin/pyi-makespec
/usr/local/bin/pyi-set_version
/usr/local/bin/pyinstaller
/usr/local/lib/python3.7/dist-packages/PyInstaller/*
/usr/local/lib/python3.7/dist-packages/pyinstaller-5.10.0.dist-info/*
Proceed (Y/n)? y
Successfully uninstalled pyinstaller-5.10.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

$ sudo pip3 install git+https://github.com/pyinstaller/pyinstaller.git

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting git+https://github.com/pyinstaller/pyinstaller.git
Cloning https://github.com/pyinstaller/pyinstaller.git to /tmp/pip-req-build-6iuefdqc
Running command git clone --filter=blob:none --quiet https://github.com/pyinstaller/pyinstaller.git /tmp/pip-req-build-6iuefdqc
Resolved https://github.com/pyinstaller/pyinstaller.git to commit 3dadde8
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: importlib-metadata>=1.4 in /usr/local/lib/python3.7/dist-packages (from pyinstaller==5.10.0) (4.11.1)
Requirement already satisfied: pyinstaller-hooks-contrib>=2021.4 in /usr/local/lib/python3.7/dist-packages (from pyinstaller==5.10.0) (2023.0)
Requirement already satisfied: setuptools>=42.0.0 in /usr/local/lib/python3.7/dist-packages (from pyinstaller==5.10.0) (67.6.0)
Requirement already satisfied: altgraph in /usr/local/lib/python3.7/dist-packages (from pyinstaller==5.10.0) (0.17)
Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=1.4->pyinstaller==5.10.0) (3.7.4.3)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=1.4->pyinstaller==5.10.0) (3.7.0)
Building wheels for collected packages: pyinstaller
Building wheel for pyinstaller (pyproject.toml) ... done
Created wheel for pyinstaller: filename=pyinstaller-5.10.0-py3-none-any.whl size=2216367 sha256=7c3ddaf326906418b1f1b8aa0a94e666863d55d6b98a31e6421b82aa8711631a
Stored in directory: /tmp/pip-ephem-wheel-cache-cs2iax0n/wheels/ac/73/37/773e265f4e1793ab70a325a5541a817ab473c32830d32e8087
Successfully built pyinstaller
Installing collected packages: pyinstaller
Successfully installed pyinstaller-5.10.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

$ pyinstaller --version
5.10.0

$ sudo pyinstaller -c -F --key 'MySecuredKey' --clean --upx-dir=/usr/bin/ /home/pi/PyInstallerTest/BusErrorTest.py

409 DEPRECATION: Bytecode encryption will be removed in PyInstaller v6. Please remove your --key=xxx argument to avoid breakages on upgrade. For the rationale/alternatives see #6999
410 INFO: PyInstaller: 5.10.0
410 INFO: Python: 3.7.3
414 INFO: Platform: Linux-6.1.23-v8+-aarch64-with-debian-10.13
416 INFO: wrote /home/pi/PyInstallerTest/BusErrorTest.spec
425 INFO: UPX is available.
426 INFO: Removing temporary files and cleaning cache in /root/.cache/pyinstaller
438 DEPRECATION: Bytecode encryption will be removed in PyInstaller v6. Please remove cipher and block_cipher parameters from your spec file to avoid breakages on upgrade. For the rationale/alternatives see #6999
440 INFO: Extending PYTHONPATH with paths
['/home/pi/PyInstallerTest']
1084 INFO: Will encrypt Python bytecode with provided cipher key
1085 INFO: checking Analysis
1086 INFO: Building Analysis because Analysis-00.toc is non existent
1086 INFO: Initializing module dependency graph...
1094 INFO: Caching module graph hooks...
1131 INFO: Analyzing base_library.zip ...
3869 INFO: Loading module hook 'hook-encodings.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
5445 INFO: Loading module hook 'hook-pickle.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
5740 INFO: Loading module hook 'hook-heapq.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
9260 INFO: Caching module dependency graph...
9657 INFO: running Analysis Analysis-00.toc
9757 INFO: Analyzing /home/pi/PyInstallerTest/BusErrorTest.py
9762 INFO: Analyzing hidden import 'tinyaes'
9763 INFO: Processing module hooks...
9799 INFO: Looking for ctypes DLLs
9800 INFO: Analyzing run-time hooks ...
9815 INFO: Looking for dynamic libraries
10908 INFO: Looking for eggs
10909 INFO: Python library not among binary dependencies. Performing additional search...
11086 INFO: Using Python library /lib/arm-linux-gnueabihf/libpython3.7m.so.1.0
11092 INFO: Warnings written to /home/pi/PyInstallerTest/build/BusErrorTest/warn-BusErrorTest.txt
11136 INFO: Graph cross-reference written to /home/pi/PyInstallerTest/build/BusErrorTest/xref-BusErrorTest.html
11160 INFO: checking PYZ
11161 INFO: Building PYZ because PYZ-00.toc is non existent
11161 INFO: Building PYZ (ZlibArchive) /home/pi/PyInstallerTest/build/BusErrorTest/PYZ-00.pyz
11718 INFO: Building PYZ (ZlibArchive) /home/pi/PyInstallerTest/build/BusErrorTest/PYZ-00.pyz completed successfully.
11723 INFO: checking PKG
11723 INFO: Building PKG because PKG-00.toc is non existent
11723 INFO: Building PKG (CArchive) BusErrorTest.pkg
15817 INFO: Building PKG (CArchive) BusErrorTest.pkg completed successfully.
15821 INFO: Bootloader /usr/local/lib/python3.7/dist-packages/PyInstaller/bootloader/Linux-32bit-arm/run
15821 INFO: checking EXE
15821 INFO: Building EXE because EXE-00.toc is non existent
15821 INFO: Building EXE from EXE-00.toc
15822 INFO: Copying bootloader EXE to /home/pi/PyInstallerTest/dist/BusErrorTest
15822 INFO: Appending PKG archive to custom ELF section in EXE
15863 INFO: Building EXE from EXE-00.toc completed successfully.

$ sudo ./dist/BusErrorTest
Bus error

[gaamaaresosa]

Btw, I have both zlib1g-dev and CGG already installed

$ sudo dpkg -s zlib1g-dev
Package: zlib1g-dev
Status: install ok installed
Priority: optional
Section: libdevel
Installed-Size: 398
Maintainer: Mark Brown broonie@debian.org
Architecture: armhf
Multi-Arch: same
Source: zlib
Version: 1:1.2.11.dfsg-1+deb10u2
Provides: libz-dev
Depends: zlib1g (= 1:1.2.11.dfsg-1+deb10u2), libc6-dev | libc-dev
Conflicts: zlib1-dev
Description: compression library - development
zlib is a library implementing the deflate compression method found
in gzip and PKZIP. This package includes the development support
files.
Homepage: http://zlib.net/

$ sudo dpkg -s GCC
Package: gcc
Status: install ok installed
Priority: optional
Section: devel
Installed-Size: 45
Maintainer: Debian GCC Maintainers debian-gcc@lists.debian.org
Architecture: armhf
Source: gcc-defaults (1.181+rpi2)
Version: 4:8.3.0-1+rpi2
Provides: c-compiler, gcc-arm-linux-gnueabihf (= 4:8.3.0-1+rpi2)
Depends: cpp (= 4:8.3.0-1+rpi2), gcc-8 (>= 8.3.0-1~)
Recommends: libc6-dev | libc-dev
Suggests: gcc-multilib, make, manpages-dev, autoconf, automake, libtool, flex, bison, gdb, gcc-doc
Conflicts: gcc-doc (<< 1:2.95.3)
Description: GNU C compiler
This is the GNU C compiler, a fairly portable optimizing compiler for C.
.
This is a dependency package providing the default GNU C compiler.

[rokm]

I get the bus error with onefile hello-world build in arm32v7/python:37-buster container, too. Will look into it.

[gaamaaresosa]

I get the bus error with onefile hello-world build in arm32v7/python:37-buster container, too. Will look into it.

Thank you so much for your dedication.

[rokm]

@gaamaaresosa Can you verify that #7571 fixes the problem for you? I.e., reinstall PyInstaller from that branch:

pip uninstall pyinstaller
pip install git+https://github.com/rokm/pyinstaller.git@fix-pkg-alignment

and then rebuild the application.

[gaamaaresosa]

@gaamaaresosa Can you verify that #7571 fixes the problem for you? I.e., reinstall PyInstaller from that branch:

pip uninstall pyinstaller
pip install git+https://github.com/rokm/pyinstaller.git@fix-pkg-alignment

and then rebuild the application.

Great !
It works fine on my oneline test.

$ pyinstaller --version 5.10.0
$ sudo pyinstaller -c -F --key 'MySecuredKey' --clean --upx-dir=/usr/bin/ /home/pi/PyInstallerTest/BusErrorTest.py
$ sudo ./dist/BusErrorTest
This is a PyInsyaller 5.10 Bus error test

Thank you so much for your prompt help !
I will test my production application also in a while.

[gaamaaresosa]

You are the best person to ask my doubts on PyInstaller.

I am very much skeptical on the security of my python binaries.
I need at least 75% tamper proof on my binaries.
We spend years on coding and the crackers easily cracks it in no time.
That's why I don't prefer Dot.NET binaries.

How far PyInstaller is safe ?
I set --key = '16chars'
What else I could follow to make it anti reversable ?

[rokm]

You are the best person to ask my doubts on PyInstaller.

I am very much skeptical on the security of my python binaries. I need at least 75% tamper proof on my binaries. We spend years on coding and the crackers easily cracks it in no time. That's why I don't prefer Dot.NET binaries.

How far PyInstaller is safe ? I set --key = '16chars' What else I could follow to make it anti reversable ?

Actually, --key does very little to protect your code, because knowing PyInstaller's internal structures (which are not a secret), it is quite easy to extract the encryption key and decrypt the bytecode-compiled modules. You could automate the procedure, and there used to be a tool that did exactly that. Therefore, we plan to remove that feature altogether - see #6999.

If you really want to obfuscate your code, either cythonize the critical modules (although be aware that PyInstaller won't be able to analyze them, so you'll have to deal with hidden imports) or use pyarmor.

[gaamaaresosa]

If you really want to obfuscate your code, either cythonize the critical modules (although be aware that PyInstaller won't be able to analyze them, so you'll have to deal with hidden imports) or use pyarmor.

Thanks again !

1. You mean the --key is ignored on the latest PyInstaller even though we pass it ?
   Why can't you retain this as one extra layer of protection at least for newbie cracker ?

2. If I pyarmor my .py files and the call PyInstaller build, which is work properly ?
   Or should we use any special trick PyInstaller build ?

[rokm]

If you really want to obfuscate your code, either cythonize the critical modules (although be aware that PyInstaller won't be able to analyze them, so you'll have to deal with hidden imports) or use pyarmor.

Thanks again !

1. You mean the --key is ignored on the latest PyInstaller even though we pass it ?

The latest version still uses key and applies the encryption, but it emits the deprecation warning. And passing --key will raise an error once the option is finally removed.

   Why can't you retain this as one extra layer of protection at least for newbie cracker ?

Because the decryption key is plain to see for anyone who knows where to look for it. And like I said, it is trivial to write a script to unpack and decrypt everything. So at the end of the day, all the --key option does is give people unfamiliar with its working a false sense of security.

2. If I pyarmor my .py files and the call PyInstaller build, which is work properly ?
   Or should we use any special trick PyInstaller build ?

I have no experience with pyarmor, really. But I think it modifies your spec file, so that PyInstaller does its import analysis on original .py files (which it can analyze), and then the modules are swapped with their pyarmor-ed counterparts in the .spec file. I imagine pyarmor describes the process in their documentation.

[gaamaaresosa]

I have no experience with pyarmor, really. But I think it modifies your spec file, so that PyInstaller does its import analysis on original .py files (which it can analyze), and then the modules are swapped with their pyarmor-ed counterparts in the .spec file. I imagine pyarmor describes the process in their documentation.

As you said currently PyIntaller scans all the coded and warn if any issues on it.
Since pyarmor is obfuscated code PyInstaller may not understand it and throw errors.
My question is can't be bypass the ScanCode function in PyInstaller with some oprion while building.
I think again while running that obfuscated code PyInstaller needs pyarmor code too.

Today's world everyone needs a good anti-reversable protection binary.
Not necessarily100% safe, even 50% will do.

[gaamaaresosa]

Because the decryption key is plain to see for anyone who knows where to look for it. And like I said, it is trivial to write a script to unpack and decrypt everything. So at the end of the day, all the --key option does is give people unfamiliar with its working a false sense of security.

Yes its a challenge to hide any secrete code now a days.

I feel very bad when the community grows in developing useless new technologies fails to concentrate of hiding secret keys ets. These pain can be understood only by the develpers who work hard for their everyday bread and butter.

You must never diclare the schema of some secrets even on opensource, especially on security related codes.
Else no such projects.

Do you think 'steganography' also be cracked ?