3.6.6 silent crash on windows (packed by pyinstaller)

[rpodgorny]

we're using orjson within a pretty-complex application (along with pyside6 and other huge libraries). the final product is being packaged for windows using pyinstaller.

lately, our app fails to start. there's some cryptic message about application crash in windows event log. after investigating, it seems the crash is caused by orjson-3.6.6 - when built with 3.6.5 (everything else being the same), it runs correctly.

i don't have much time to investigate further right now. hopefully you'll have the idea what might be wrong... ...if not, i'll try to provide more info later...

[ijl]

If you find an issue with your custom build setup traced to this library I will accept a patch.

[ThirVondukr]

@ijl
I've also encountered this issue with orjson==3.6.6 in my project, but for some reason i wasn't able to reproduce crash with this specific version, in separate project it crashes with 3.6.* (i didn't test versions under 3.6), versions of ojrson.
I hope you would be able to reproduce it: https://github.com/ThirVondukr/pyinstaller-orjson

[himbeles]

@ijl
I've also encountered this issue with orjson==3.6.6 in my project, but for some reason i wasn't able to reproduce crash with this specific version, in separate project it crashes with 3.6.* (i didn't test versions under 3.6), versions of ojrson.
I hope you would be able to reproduce it: https://github.com/ThirVondukr/pyinstaller-orjson

I can reproduce the bug with the code in the linked repo. On Windows 10, Python 3.9.10 and Python 3.10.2, pyinstaller 4.8 & 4.9 . Silent crash on import.

[Traktormaster]

Version 3.6.6 also crashes an application for me. I'm using Python 3.9.9 on macOS 10.14. It's very strange because the crash only happens when the application is run from an application bundle. When I'm running it from the virtualenv it works as expected. I'm using cx_Freeze to create the app btw.

If I downgrade to version 3.6.5 the built app works.

Edit here is the top of the crash report:

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000008
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [12062]

VM Regions Near 0x8:
--> 
    __TEXT                 000000010d092000-000000010d094000 [    8K] r-x/rwx SM=COW  /Users/USER/*/AppX.app/Contents/MacOS/AppX

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   Python                        	0x000000010d142abd PyObject_GenericGetDict + 6
1   orjson.cpython-39-darwin.so   	0x000000010e6b9b99 0x10e662000 + 359321
2   orjson.cpython-39-darwin.so   	0x000000010e684305 0x10e662000 + 140037
3   orjson.cpython-39-darwin.so   	0x000000010e6bd74b 0x10e662000 + 374603
4   orjson.cpython-39-darwin.so   	0x000000010e6ba09c PyInit_orjson + 1219
5   Python                        	0x000000010d231905 _imp_create_dynamic + 1360
6   Python                        	0x000000010d14c8d8 cfunction_vectorcall_FASTCALL + 95
7   Python                        	0x000000010d1fe008 _PyEval_EvalFrameDefault + 45427
8   Python                        	0x000000010d1f1d1d _PyEval_EvalCode + 391
9   Python                        	0x000000010d10146d _PyFunction_Vectorcall + 377
10  Python                        	0x000000010d2004c1 call_function + 167
...

[himbeles]

For me, a pyinstaller hook for hidden stdlib imports and numpy import fixed the issue.
I have contributed it as PR to https://github.com/pyinstaller/pyinstaller-hooks-contrib:

https://github.com/pyinstaller/pyinstaller-hooks-contrib/blob/08dbcf24a798f43ac3a373f89cee081855589675/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-orjson.py

The problem is the forced import of modules in src/typeref.rs, which pyinstaller doesn't know about.

[ThirVondukr]

@himbeles I'm curious what is has to do with numpy 🤔

[himbeles]

orjson supports numpy encoding/decoding. On orjson import, it tries to load the numpy library to detect different types it could encounter during encoding/decoding. Adding numpy to the hook hidden imports has no effect if numpy is not installed in your virtual environment. Pyinstaller will not crash, but instead just not bundle it in the executable and throw a warning. Have you tried if the hook works for you?

[himbeles]

throw the hook-orjson.py file into a hooks directory and call pyinstaller with the --additional-hooks-dir=hooks argument.

[ThirVondukr]

Not yet tried it, would be able to in a couple of days, i just downgraded orjson to 3.6.5 for now

[himbeles]

Hey, my PR got merged into pyinstaller last week. You can update pyinstaller-hooks-contrib to get the new hook.