Using codesigning flags still results in "no cdhash, completely unsigned?" errors (with --onedir executable or app bundle executable) #8029

petiatil · 2023-10-22T01:16:26Z

After researching similar issues extensively for a few days without a solution, I would be grateful to receive any level of guidance.

My use case:

macOS Ventura 13.6

In Swift, invoke the (6.1.0) PyInstaller-generated executable with Process()
- Hangs since the executable has been signed (Error details posted below)
- Side note: An Info.plist file is not currently associated with the executable, as these tests haven't been with a PyInstaller-generated app bundle.

Codesigning flags used:

"--codesign-identity", identity # "Apple Development: My Name (..........)"
"--osx-entitlements-file", entitlements_path # "/path/to/app_entitlements.entitlements"
"--osx-bundle-identifier", bundleID # "com.appname.macos"

Command structure (via Python script):

pyinstaller_command = [
    "/Users/.../3.10 venv/bin/python3.10", "-m", "PyInstaller", "--onedir", "--additional-hooks-dir=my_hooks",
    
    # ... add-data, hidden imports, copy metadata, add binaries ...

    # codesign flags here
    "app_name.py"
]

Entitlements to inherit:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.app-sandbox</key>
    <true/>
    <key>com.apple.security.inherit</key>
    <true/>
</dict>
</plist>

Main app entitlements:

com.apple.security.app-sandbox, com.apple.security.cs.disable-library-validation, among others (at least temporarily, such as com.apple.security.cs.allow-unsigned-executable-memory, com.apple.security.cs.allow-dyld-environment-variables, and com.apple.security.cs.allow-jit)

Error samples from the Console app:

 error 19:42:36.217483-0500 kernel Sandbox: app_name(22883) deny(1) ipc-posix-sem-create /joblib-22883-aagrqso7
...
error 19:42:37.498106-0500 kernel Sandbox: app_name(22883) deny(1) network-bind local:*:0

error 19:42:37.614221-0500 kernel Library Validation failed: Rejecting '/private/var/folders/z4/snz0v2gd3z1__qglgqc09bm00000gn/T/com.appname.macos/ffiMINHOs' (Team ID: none, platform: no) for process 'app_name(22883)' (Team ID: MyTeamID, platform: no), reason: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.
...
error 19:42:37.614737-0500 syspolicyd Disallowing load of in 22883,
error 19:42:37.615163-0500 kernel Sandbox: app_name(22883) deny(1) file-write-create /private/tmp/ffikQWqG4
error 19:42:37.615228-0500 kernel Sandbox: app_name(22883) deny(1) file-write-create /private/var/tmp/ffipjwZqi

What I've tried

All combinations of codesigning (signing root-first (top-level last), with and without various flags, doing so manually without PyInstaller codesign flags, with flags and manually, etc.)
- And importing into Xcode's "Copy Files" build phase with and without "code sign on copy"

e.g. when signing manually:

result = subprocess.run(
     ["codesign", "--deep", "--force", "-s", identity, "-i", bundleID, "-o", "runtime", "--entitlements",
      entitlements_path, "--timestamp", "-f", filepath], capture_output=True, text=True)

What I'm considering

testing with a PyInstaller-generated app bundle* rather than testing directly with the executable
*(from Swift: launching the executable within the app bundle instead of the --onedir executable)

If anyone has faced similar issues, or has insights into why the codesigning flags might not be working, I'd greatly appreciate any advice.

The text was updated successfully, but these errors were encountered:

rokm · 2023-10-22T09:47:45Z

When (re)signing manually, did you (re)sign every binary (dylib, executable, .framework bundle) in the PyInstaller-generated onedir application?

For the time being, you should replace your actual PyInstaller-frozen application with a basic print("Hello world") application, to make it easier to debug (i.e., fewer binaries that need to be signed).

Does it work with PyInstaller 6.x and hello-world program? What if you switch to PyInstaller 5.13.2?

In 6.x, we are preserving the structure of .framework bundles from which we collect dylibs (so in the hello-world, that would be Python.framework at least), but in POSIX builds, we do not (re)sign the collected .framework bundle (only the dylib in it). When generating .app bundles, we can sort of get away with it because we force recursive (re)signing of the whole .app via --deep, but in POSIX builds, the unsigned .framework bundles might be causing the problem.

petiatil · 2023-10-24T01:08:52Z

@rokm Minor update: I can confirm that the basic "hello world" application works, and I do (re)sign every binary. Thank you for your help. I've narrowed the issue to an import statement (import whisperx). That module is associated with libtorch_cpu.dylib, which seems to be where the App Sandbox encounters a problem.

(Revealed by Open Files and Ports in Activity Monitor, for reference):

/private/var/folders/z4/snz0v2gd3z1__qglgqc09bm00000gn/T/TemporaryItems/NSIRD_XCBBuildService_XnaOZR/CleanBuildFolderInProgress/Debug/App Name.app/Contents/Resources/python/app_name/libtorch_cpu.dylib
/dev/ttys000
->0x9a3dc64d46386abc
->0xb48e93a4e9d079e7

# cycles through attempts like the following on a loop, but are being blocked.

/private/var/folders/z4/snz0v2gd3z1__qglgqc09bm00000gn/T/com.appname.macos/ffih5k3eh
/Users/name/Library/Containers/com.appname.macos/Data/ffiZVbXt7
/private/var/folders/z4/snz0v2gd3z1__qglgqc09bm00000gn/T/com.appname.macos/ffii7Ze0k

Side note: I double-checked and believe that I've already modified all dynamic links of libtorch_cpu.dylib (among others, with install_name_tool) to link locally. However, again, the issue is apparently linked to something not codesigned. I'm uncertain if the non-signed source is within my directories (or the PyInstaller executable) or if the source isn't signed because it might be dynamically created/loaded from another location.

I switched to 5.13.2, to be sure, and had the same issue.

I plan to test with an .app bundle if no other solution arises.

I'd appreciate any further insights on how to tackle this issue. I'll post back with any progress.

rokm · 2023-10-24T09:23:58Z

Could be that torch JIT is blocked - try looking into additional entitlements for run-time exceptions. com.apple.security.cs.allow-jit and com.apple.security.cs.allow-unsigned-executable-memory are likely candidates.

petiatil · 2023-10-24T13:01:52Z

Shortly after my initial post, I updated it to display my "Main app entitlements" rather than just the 'inherit' entitlements. To be clear, I've continued to use both the 'allow-jit' and 'unsigned memory' entitlements (based on some posts I've come across).

I wasn't aware of the torch / JIT connection, though. I'll keep that in mind, thank you.

Hopefully using an app bundle will solve it (if the POSIX approach is why something wasn't being signed - whether due to JIT or otherwise). If not, it appears I may need to figure out how to use a static build of libtorch_cpu.

petiatil · 2023-10-24T14:00:22Z

To conclude my previous comment:

I'll proceed to test with an app bundle. Unless advised differently, I'll start with version 5.13.2.
My current assumption is that "additional-hooks" might not be the best place for manual codesigning. At the moment, any manual codesigning is done after the entire process is finished.

rokm · 2023-10-24T14:04:31Z

I'm proceeding to try to test with an app bundle. As I'm not sure what version will be most recommended for this case, I'll start with 5.13.2, unless directed otherwise.

For .app bundles, v6 is actually recommended as it improves the bundle layout for codesigning compliance.

My current assumption is that "additional-hooks" isn't a place where any manual codesigning might best occur. Any manual codesigning is currently after the entire process completes.

Indeed. "Additional hooks" are involved only during analysis (designating what needs to be collected and where). Built-in signing is performed during actual collection (for individual binaries) and then also once the whole .app bundle is assembled (this part is recursive, in an attempt to ensure that everything is signed). Any manual re-signing must be done in post-processing, outside of PyInstaller.

petiatil · 2023-10-25T18:29:44Z

Results from testing with an app bundle:

In the codesigning phase, an error occurs, which may or may not relate to the main issue:

'''
1376 INFO: PyInstaller: 6.1.0
1376 INFO: Python: 3.10.6
1411 INFO: Platform: macOS-13.6-x86_64-i386-64bit
...
222686 INFO: Re-signing the EXE
223591 INFO: Building EXE from EXE-00.toc completed successfully.
223872 INFO: checking COLLECT
223872 INFO: Building COLLECT because COLLECT-00.toc is non existent
223872 INFO: Building COLLECT COLLECT-00.toc
258711 INFO: Building COLLECT COLLECT-00.toc completed successfully.
259891 INFO: checking BUNDLE
259891 INFO: Building BUNDLE because BUNDLE-00.toc is non existent
259891 INFO: Building BUNDLE BUNDLE-00.toc
302965 INFO: Signing the BUNDLE...
347337 WARNING: Error while signing the bundle: codesign command (['codesign', '-s', 'Apple Development: My Name (..........)', '--force', '--all-architectures', '--timestamp', '--options=runtime', '--entitlements', '/Users/name/App Name (MacOS)/app_entitlements.entitlements', '--deep', '/Users/name/App Name (MacOS)/dist/app_name.app']) failed with error code 1!
output: /Users/name/App Name (MacOS)/dist/app_name.app: replacing existing signature
/Users/name/App Name (MacOS)/dist/app_name.app: .DS_Store files cannot be a symlink

347338 WARNING: You will need to sign the bundle manually!
347338 INFO: Building BUNDLE BUNDLE-00.toc completed successfully.
'''

I proceed to remove all .DS_Store files and codesign everything manually (iterating directories root-first):

result = subprocess.run(
    ["codesign", "--deep", "--force", "-s", identity, "-i", "com.appname.macos", "-o", "runtime", "--entitlements",
     entitlements_path, "--timestamp", "-f", filepath], capture_output=True, text=True)

In a separate test, I also tried signing just the app file (with "--deep", etc).

As the same "cdhash, completely unsigned?" issue persists:

Hopefully the issue is simply that the manual codesigning can't reach the PyInstaller executable contents (as the executable is already compiled at the time of manual codesigning), where the solution might then just be to solve the DS_Store files error in the PyInstaller process.

rokm · 2023-10-25T20:02:24Z

Where did the .DS_Store file(s) come from?

rokm · 2023-10-25T20:07:02Z

As the same "cdhash, completely unsigned?" issue persists:
Hopefully the issue is simply that the manual codesigning can't reach the PyInstaller executable contents (as the executable is already compiled at the time of manual codesigning), where the solution might then just be to solve the DS_Store files error in the PyInstaller process.

That doesn't seem to be the case to me. If you removed .DS_Store files and (re)signed the bundle, then you should get same result as if .DS_Store files were not an issue.

So if the "cdhash, completely unsigned?" issue persists, you will have to figure out which part is unsigned and is causing issues. Maybe you could try packaging torch but not actually using it (i.e., do a conditional import), to rule out any JIT-related issues.

rokm · 2023-10-25T20:09:22Z

Similarly, you could try enabling sandbox on the .app bundle and see if stand-alone .app bundle runs as expected after signing it (i.e., to see if issues also occur without the sandbox inheritance).

rokm · 2023-10-25T20:22:10Z

One other thing - you never mentioned what packages your application is using nor what your python environment is (anaconda, python.org+pypi, homebrew)...

Can you run otool -l <path/to/binary> | grep -B 1 -A 3 LC_VERSION_MIN_MACOSX against all binaries in the .app bundle, and see if any reports sdk n/a (starting perhaps with the most obvious candidate, libtorch_cpu.dylib)?

rokm · 2023-10-25T21:38:50Z

Can you run otool -l <path/to/binary> | grep -B 1 -A 3 LC_VERSION_MIN_MACOSX against all binaries in the .app bundle, and see if any reports sdk n/a (starting perhaps with the most obvious candidate, libtorch_cpu.dylib)?

sklearn/.dylibs/libomp.dylib is also a likely candidate (at least for python.org + pypi).

petiatil · 2023-10-26T03:33:33Z

.DS_Store locations:

app_name.app/Contents/Resources/pyphen
app_name.app/Contents/Resources/nltk_data/tokenizers/punkt
app_name.app/Contents/Resources/speechbrain
app_name.app/Contents/Resources/nltk
app_name.app/Contents/Resources/torch
app_name.app/Contents/Resources/torch/include
app_name.app/Contents/Resources/torch/include/torch/csrc
app_name.app/Contents/Resources/torch/include/torch/csrc/jit
app_name.app/Contents/Resources/torch/include/ATen
app_name.app/Contents/Resources/torch/utils
app_name.app/Contents/Resources/torch/share/cmake
app_name.app/Contents/Resources/sklearn
app_name.app/Contents/Resources/transformers
app_name.app/Contents/Resources/numba
app_name.app/Contents/Frameworks/torch
app_name.app/Contents/Frameworks/sklearn
app_name.app/Contents/Frameworks/numba

Python environment

I created my virtual Python environment using the standard Python installation from python.org. I am currently using Python version 3.10.6, which was compiled with Clang 13.0.0.

Some Python packages of the application:

whisperx, whisper (just as whisperx, triggers the same issue: on import in the python script, the process (shown as libtorch_cpu.dylib in Open Files and Ports) continually tries to load some temporary element and is continually denied, so it doesn't move beyond that import statement), torch, ffmpeg, among some unrelated others.

Other PyInstaller command details, if useful to know:

... "--add-data", "/Users/name/App Name (MacOS)/3.10 venv/lib/python3.10/site-packages/whisper/assets:whisper/assets",
"--hidden-import=pytorch",
"--collect-data", "torch",
"--copy-metadata", "torch",
"--copy-metadata", "tqdm",
"--copy-metadata", "regex",
"--copy-metadata", "sacremoses",
"--copy-metadata", "requests",
"--copy-metadata", "packaging",
"--copy-metadata", "filelock",
"--copy-metadata", "numpy",
"--copy-metadata", "tokenizers",
"--copy-metadata", "importlib_metadata",
"--copy-metadata=rich",
"--hidden-import=speechbrain",
"--hidden-import=rich",
"--hidden-import=sklearn.utils._cython_blas",
"--hidden-import=sklearn.neighbors.typedefs",
"--hidden-import=sklearn.neighbors.quad_tree",
"--hidden-import=sklearn.tree",
"--hidden-import=sklearn.tree._utils",
"--hidden-import=transformers.models.wav2vec2.processing_wav2vec2",
"--hidden-import=transformers",
"--collect-data=transformers",
"--copy-metadata=transformers",

# The following are included in the app bundle to prevent attempting their access on the system (for compatibility across systems)
"--add-binary", "/dylib/location//dylibs/libogg.0.dylib:.", 
"--add-binary", "/dylib/location//dylibs/libvorbis.0.dylib:.",
"--add-binary", "/dylib/location//dylibs/libsndfile.1.0.35.dylib:.",
"--add-binary", "/dylib/location//dylibs/libFLAC.12.dylib:.",
"--add-binary", "/dylib/location//dylibs/libopus.0.dylib:.",
"--add-binary", "/dylib/location//dylibs/libvorbisenc.2.dylib:.",
"--add-binary", "/dylib/location//dylibs/libmpg123.0.dylib:.",
"--add-binary", "/dylib/location//dylibs/libmp3lame.0.dylib:.",
"--add-binary", "/dylib/location//dylibs/libxcb.1.dylib:.",
"--add-binary", "/dylib/location//dylibs/libxcb-shm.0.dylib:.",
"--add-binary", "/dylib/location//dylibs/libxcb-shape.0.dylib:.",
"--add-binary", "/dylib/location//dylibs/libxcb-xfixes.0.dylib:.",
"--add-binary", "/dylib/location//dylibs/libSDL2-2.0.0.dylib:.",
"--add-binary", "/dylib/location//dylibs/liblzma.5.dylib:.",
"--add-binary", "/dylib/location//dylibs/libx264.164.dylib:.",
"--add-binary", "/dylib/location//dylibs/libx265.199.dylib:.",
"--add-binary", "/dylib/location//dylibs/libX11.6.dylib:.",

Regarding the otool -l <path/to/binary> | grep -B 1 -A 3 LC_VERSION_MIN_MACOSX test, nothing was printed for any of the binaries (in the Frameworks folder).

All works well until signing the executable with the sandboxing/inherit entitlements. It also works if all associated binaries are signed as long as the executable isn't signed (last I checked).

Similarly, you could try enabling sandbox on the .app bundle and see if stand-alone .app bundle runs as expected after signing it (i.e., to see if issues also occur without the sandbox inheritance).

I'm having trouble understanding this request. Could you please clarify? (Otherwise, I'll think more on it when fresh).

Side note: If of any use, at your request I could also post the full list of signed/validated binaries (printed from codesign --verify --verbose=4 /Users/name/App\ Name\ \(MacOS\)/app_name.app)

rokm · 2023-10-26T20:20:19Z

.DS_Store locations:

...

I see those are created by Finder when you navigate the package directories, and we shouldn't really be collecting them at all (#8042).

Some Python packages of the application:

whisperx, whisper (just as whisperx, triggers the same issue: on import in the python script, the process (shown as libtorch_cpu.dylib in Open Files and Ports) continually tries to load some temporary element and is continually denied, so it doesn't move beyond that import statement), torch, ffmpeg, among some unrelated others.

Can you capture/copy/retrieve that temporary element somehow, so we can look at what it is? Coming from libtorch_cpu.dylib, I still suspect it is part of torch JIT. Admittedly, I never really looked into details of it, but if JIT generates binaries, it is unreasonable to expect that they will be signed with your team ID (though perhaps ad-hoc signing would not be unexpected). So it might be that torch is inherently incompatible with macOS hardened run-time.

Regarding the otool -l <path/to/binary> | grep -B 1 -A 3 LC_VERSION_MIN_MACOSX test, nothing was printed for any of the binaries (in the Frameworks folder).

Hmmm, I'd expect sklearn/.dylibs/libomp.dylib to be problematic (since I see whisperx pulls in scikit-learn). Although that's probably the case only withx86_64 PyPI wheels (are you on arm64?). Either way, libomp does not to be an issue here (yet?), but I mentioned it because it came up in another discussion about macOS and hardened runtime (#7899).

Either way, #8043 will add a warning about potentially problematic libraries.

All works well until signing the executable with the sandboxing/inherit entitlements. It also works if all associated binaries are signed as long as the executable isn't signed (last I checked).

Similarly, you could try enabling sandbox on the .app bundle and see if stand-alone .app bundle runs as expected after signing it (i.e., to see if issues also occur without the sandbox inheritance).

I'm having trouble understanding this request. Could you please clarify? (Otherwise, I'll think more on it when fresh).

It is unclear what exactly you are testing with right now - are you still testing with PyInstaller-generated .app bundle as a helper application to another application? So, does PyInstaller .app bundle still inherit sandbox from the main application?

My suggestion was to test PyInstaller .app bundle as a stand-alone application, with its own sandbox (and thus no entitlements inheritance, but rather its own set of entitlements) - just to make sure that this is not some sort of issue with entitlements inheritance.

Side note: If of any use, at your request I could also post the full list of signed/validated binaries (printed from codesign --verify --verbose=4 /Users/name/App\ Name\ \(MacOS\)/app_name.app)

I don't think that would help at this point.

petiatil · 2023-10-27T23:45:31Z

The temporary elements

I'm pleased to report (in hopes that the issue is much simpler than first deemed) that 'hanging at the whisperx import' was incidental. It just happened to land there, as it has a lot to import. But it will hang irrespective of a particular import. What I found in "Open Files and Ports" in, otherwise, successful tests (described below) is that the next item after libtorch_cpu.dylib is "/Users/name/App Name (MacOS)/app_name.app/Contents/Resources/base_library.zip". That is, the libtorch_cpu dylib 'open file' incidentally preceded the loading of what is apparently PyInstaller's base_library.zip (the 'open file' that displays next, which makes sense to be the actual unidentified temporary elements that were being halted, and seems to make sense of why the process had been failing only after signing the executable (as I presume it's trying to load or unpack base_library.zip).

When testing the app bundle directly (launching its POSIX executable from Terminal, rather than as a helper process launched from the parent Swift app):
- As you suggested for a test, the full entitlements that I use for the parent (Swift) app were used to codesign (rather than the 'inherit' entitlements)
- It seems to run properly, getting passed all imports.
  - This is after manually codesigning the app bundle (with arguments: ["codesign", "--force", "--deep", "-s", identity, "-i", "com.appname.macos", "-o", "runtime", "--entitlements", entitlements_path, "--timestamp", "-f", appBundlePath])`
    - Note: When I verify the codesignature for the POSIX executable (in the app bundle), it seems to print verification statements as if I verified the app bundle itself (which I intuit may be a good sign for inheritance compatibility, as again the issue has related to signing the executable itself), but I wasn't expecting it to link to all of the framework files).
    - Note: I assume that entitlements are working when testing the app bundle directly, but if needed, I plan to confirm this with a manual signing process that doesn't use deep (but signs the inner-most first of each directory)
- If I understand correctly, I only need to manually sign with one line (the app bundle, using the deep flag, etc, or the POSIX-only signing the directory).  
  - Side note: For the record, until today, I've mostly been signing everything manually (usually without --deep, since I'm iterating the directories manually), but essentially with the same result

My system and what I'm testing with

All is built from a 2019 MacBook Pro (x86_64), Ventura version 13.6.1

Side note: The virtual environment was created in the standard way using Python (Not sure that was clear).

It is indeed a helper executable and the bundle inherits the sandbox, used as follows from a Swift-based application:

if let pythonExecutablePath = Bundle.main.path(forResource: "python/app_name.app/Contents/MacOS/app_name", ofType: "")
// ...executable path if testing with the app bundle

but getting the same result whether I test with the app bundle executable or the 'onedir' executable.

Next steps

If/when you deem the following a priority given the latest info:

I want to ensure the issue doesn't relate to some sort of inconsistent signature detection.
- Therefore, preferably I could test with a PyInstaller version that removes all .DS_Store files so it can complete its signature process without my overriding signature process; As that might not override everything signed by the PyInstaller process (for anything compiled in the executable, for instance), maybe it could lead to an inconsistency detected by the Xcode build (of the parent app) that launches the helper process.

Side notes (Feel free to ignore until ever apt):

If/when you think that reaching out to Apple developers (using one of my two credits for this year) is in order, feel free to let me know.
If/when it's determined that the issue is actually torch-related, I have verbose output of the whisperx Python import
If needed, I'll also plan to test with the directory created alongside the app bundle (that looks like a 'onedir' build)

rokm · 2023-10-28T21:49:30Z

If problem was related to base_library.zip, I would imagine your earlier test with hello-world PyInstaller application to fail as well (since it should contain an identical copy). In addition, base_library.zip contains python modules that are loaded and used during bootstrap (the .zip is not unpacked anywhere; python reads it using built-in zip reader), so if that was blocked for any reason, the frozen application would not even reach your program code, but would fail either during interpreter initialization, or while running our bootstrap scripts and/or run-time hooks. And lastly, if this was really the problem in whatever way, it would likely be observed by anyone using hardened runtime with their macOS app...

petiatil · 2023-10-29T01:40:26Z

Good points to reflect on, thank you. To clarify (if I understand upon further reflection), and to your point on the hello-world app, I don't believe the problem is inherent to base_library.zip, but perhaps to a codesigning discrepancy detected by Apple / the parent app (i.e., not merely verify). Perhaps this discrepancy is occurring through the conjunction of inheritance* codesigning and a mutually incompatible code-signing process [1]

*In contrast to full entitlement signing (testing with the app directly) - Because that bifurcated approach (manually signing what I can after PyInstaller partially signs, stopped by .DS_Store error) works in that case.

[1] That is, to the extent my signature process may diverge from PyInstaller's, such as:

The codesignature/arguments themselves [?]
- having to manually codesign (with a divergent signature [?]) after PyInstaller doesn't complete the process (due to .DS_Store error)
not being able to sign PyInstaller-compiled elements (those within the executable in the app bundle).
- (when manually codesigning with the potentially divergent signature after .DS_store error)
And I assume the issue doesn't relate to something in the executable not being signed by PyInstaller.

Again, this feels like good news [2] after the apparent successful test without inheritance entitlements - instead, using the exact entitlements the parent app uses and testing directly [3] rather than as a helper process.

[2] because it would indicate that the app is indeed sandbox compatible in its current state (so not related to torch intricacies), and the issue likely relates to how inheritance codesigning is being applied/synced.
[3] Note: The test involved launching the executable within the codesigned app bundle from Terminal (rather than inheritance codesigning and launching from Swift).

If after a test with no .DS_Store errors in the PyInstaller codesigning process yields the same result, and it's still unclear where the codesigning discrepancy is, I will prepare to contact Apple developers directly (after gathering any questions, statements, or insights you might want to impart).

Side note:

It is my current understanding that the App Sandbox restrictions should apply to the app bundle when testing it directly (i.e. Shouldn't matter that it's outside of the Xcode-built process; Codesigning with App Sandbox entitlements will restrict its access the same).

rokm · 2023-10-30T20:36:53Z

I doubt .DS_Store files are the issue here, but you can check with develop branch (or remove all .DS_Store files using find command in the shell). If you are properly (re)signing the bundle, then it shouldn't matter that PyInstaller originally failed to sign the bundle due to .DS_Store files. Since you are using --force, you are replacing any pre-existing signatures with your own, so there should be no divergence/discrepancy.

petiatil · 2023-10-31T16:20:47Z

Since you are using --force, you are replacing any pre-existing signatures with your own, so there should be no divergence/discrepancy.

Good reminder, I would just maintain skepticism with respect to any lateral differences of manual codesigning and that which I can't sign (compiled in the executable).

However, if I can actually prevent the .DS_Store error by removing files before compilation so I don't need to manually sign:

then if the same error persists, I would tentatively posit three potential sources of the issue (some shots in the dark, trying to consider everything, so feel free to consider them without an explicit reply to anything in particular):
- I am missing some implementation of appropriate codesigning configuration for PyInstaller to deploy
  - I currently have 3 codesigning flags for the PyInstaller command list, no other helper script or spec file adjustment
    - Side note: I've been erring on the side of likely-more-than-needed entitlements to start (and will remove as needed later):
      - (e.g. 'allow DYLD', 'allow unsigned memory', 'allow JIT')
- Signing with the "inherit" entitlements plist needs a different signing approach to ensure proper relation to a parent app (not stopped at its own structure [?] - recalling that something isn't being signed - perhaps due to not inheriting properly in that structure)
- The helper-launch structure from the parent app needs to be placed in executables rather than Resources, or 'create groups' rather than 'folder references' when adding to the 'Copy Files' build phase.

I will try the pre-removal of .DS_Store files and post an update.

Appreciate your input and analyses – it's helping me navigate through a critical phase.

Best regards,

Phillip

petiatil · 2023-10-31T16:50:03Z

(Message 2) Quick update: I'm not certain, but (answering my own question) it may be crucial to place the executable in Executables rather than the default Resources in the Copy Files phase. I'll report back soon.

rokm · 2023-10-31T17:38:35Z

(Message 2) Quick update: I'm not certain, but (answering my own question) it may be crucial to place the executable in Executables rather than the default Resources in the Copy Files phase. I'll report back soon.

Resources is definitely wrong place for binaries, executables, or .framework and .app bundles...

petiatil · 2023-11-02T22:05:19Z

Notes on current results (specifics in attached image, if useful).

After testing with adding the app bundle to Executables, the result is the same.

Note: The first thing that occurs is a denial of 'file-write-create' in '/private/tmp/tmpData1' and '/private/var/tmpData2'.

Do you think the issue could be resolved if the temporary data was created in the App Sandbox instead of '/private/...' (conforming to the rules so its creation wouldn't be denied)? Maybe there wouldn't be a subsequent signage error

I ask because there is technically an empty document created (which of course is unsigned) when the denial occurs, which perhaps is what triggers the subsequent errors.

NOTE

The app bundle does not have "code sign on copy" like a regular executable does. However, I manually signed the components in the app bundle after the parent app was built by Xcode, also with the same result.
I will try to test the 'onedir' approach again, keeping in mind anything you specified about any structural differences in PyInstaller's process (how it may handle codesigning or Mac-specific structure differently).

Key reiterations:

Again, the issue appears to be non-specific to any particular import as previously thought (such as whisperx), as it hangs at a different random place (e.g. from pathlib import Path) when delaying that import. So if there is some argument I can add to the PyInstaller build process that ensures nothing is loaded in a private system area, maybe that would be a pertinent area of focus.
As running the app bundle directly (signed with my main app's entitlements) seemed to work (i.e. doesn't hang due to the tmp data creation getting denied), maybe the fact it's launched as a helper process (the executable within the app bundle) has some implication on why something is being created in a place that's failing (e.g. where the non-inheritance entitlements and launching from the app directly ensure proper relation in its app structure).

I must admit, after the extensive effort I've dedicated exclusively to this endeavor (my project in general), the current obstacle is daunting and potentially disheartening. It seems that due to my limited funds, official distribution is essential for my project to enter the public domain and let alone gain traction. Reworking the business model and taking a separate ground-up approach is on my radar, but may take a significant chunk of unforeseen additional time, assuming that approach is practicable.

I'll await your next message/s to see if you have any other advice or things to test or consider. After which, contacting Apple (using a 'code support credit') might be the next step; Maybe the issue (the hurdle of integrating this sort of helper process) is familiar or unique to Apple's architecture.

What else I'm considering

if it's possible the entitlements weren't inherited in time
whether it's possible to somehow configure to launch the app bundle with its own sandbox (rather than inheriting as a helper process) from the parent app.
whether the issue could stem from launching the executable directly (rather than launching the app bundle itself as the helper, if that's possible)
if an XPC approach could apply here

petiatil · 2023-11-06T20:11:28Z

P2: Quick update:

Sleeping before importing anything in Python, to test whether entitlements weren't inherited in time, may have prevented the current error!

I'll report back with any other key updates (I still need to test further), and not to forget some comments in my previous message (if needed)

Side note:

Today I replaced the utils.py in the develop branch's "building" folder after seeing the update, but that's likely not the cause of the solution based on my current understanding, just full disclosure

petiatil · 2023-11-07T00:02:19Z

P3: Update n2:

Waiting before importing apparently wasn't necessary. I don't know what else changed besides replacing the util.py file at this point, but my tentative intuition is that it could be something else (e.g. deleting build folders, using Groups instead of folder references when adding to Copy Files, or something I'm unaware of)

Further verification to come (to ensure the sandbox is indeed being implemented)...

petiatil · 2023-11-09T03:49:19Z

I'm pleased to report that it appears that both testing and notarization have appeared to be successful with the App Sandbox

Haven't tested with the POSIX only build.
Using inherit entitlements for the app bundle.
Not sure if it had anything to do with replacing utils.py file with the recently updated version, or if it was something obvious such as ensuring build folders were deleted or using Groups when importing to Copy Files instead of folder references

Thank you for your time and input.

Best regards,

Phillip

petiatil · 2023-11-11T19:15:07Z

Update

regarding:

[1]. a seemingly outdated dylib that Apple requires to be up-to-date (when submitting to the Mac App Store)
- It seems this refers to tkinter, which I don't believe I need for the PyInstaller app (as my usage of the bundle is purely non-UI based). Is this something I can remove without causing issues?

[2]. an issue apparently to do with libomp, which you mentioned as a possible culprit earlier (screenshot of part of Apple's stack trace attached).

Quick screenshot and full file attached

ips file.txt

[1].

'''
Your app uses or references the following non-public or deprecated APIs:

Contents/MacOS/app_name.app/Contents/Frameworks/libtk8.6.dylib
Symbols:
• _NSWindowDidOrderOnScreenNotification

The use of non-public or deprecated APIs is not permitted on the App Store, as they can lead to a poor user experience should these APIs change and are otherwise not supported on Apple platforms.
'''

[2]. After submitting the current version to Apple (after successfully codesigning and running on my device), it was reported that the app crashed when attempting to run the software referred to in this post.

After reviewing some of the link you attached, where the thread ended, it's unclear to me what I might consider trying first, so if you have any specific insight or advice, it would be much appreciated.

Shot in the dark

I don't have an ARM device to test on, so I'm not sure whether running certain entitlements with the codesignature could prevent the issue (e.g. dyld-related, unsigned memory, or JIT-related)

rokm · 2023-11-11T19:56:25Z

[1] If you're not using tkinter, it is likely collected due to optional import in some part of the 3rd party code. You can use --exclude tkinter --exclude _tkinter (or add these two modules to excludes list in the spec file).

[2] Yeah, the x86_64 PyPI wheel for scikit-learn contains out-dated/problematic build of libomp.dylib - you can perhaps try swapping it with Homebrew-provided version.

petiatil · 2023-11-12T20:21:10Z

I made a mistake. Earlier I had temporarily disabled SIP (for a separate purpose), which now seems to be why the eponymous error was averted.

Your latest comments are helpful, and the eponymous error didn't occur for the Apple reviewer (but perhaps it would be their next error, not sure).

So it looks like the current status remains (at this comment)

rokm · 2023-11-12T22:10:21Z

Hmmm... if I understand correclty, https://developer.apple.com/forums/thread/120647 implies that entitlements are not inherited by the child process when it inherits the sanbox. Could this be the problem here? I.e., your child program requires com.apple.security.cs.disable-library-validation et al., but it effectively does not have them.

If you go back to the "stand-alone" version of the child program, i.e., launched on its own and with its own sandbox - does it work without any additional entitlements or not?

rokm · 2023-11-13T10:52:00Z

FWIW, in #8099 we established that one possible source of error messages like these

error 19:42:37.614221-0500 kernel Library Validation failed: Rejecting '/private/var/folders/z4/snz0v2gd3z1__qglgqc09bm00000gn/T/com.appname.macos/ffiMINHOs' (Team ID: none, platform: no) for process 'app_name(22883)' (Team ID: MyTeamID, platform: no), reason: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

are binaries that are generated by CFFI on-the-fly (and are thus lacking signature), in combination with lack of proper entitlements that would allow them to be loaded.

Which further reinforces my suspicion that in your case, the problem is that entitlements from the parent/inherited sandbox are not properly inherited by the child process, where they are required.

petiatil · 2023-11-14T02:38:34Z

I really appreciate you finding/providing that link - The solutions therein might directly align with my requirements. As per that info, I'm now considering the potential "CFMessagePort" approach (rather than something like an XPC service), if that makes sense. I'll soon look at the #8099 info as well.

Current testing notes (before testing any structural changes to the launch approach, such as CFMessagePort):  When providing a more comprehensive set of entitlements (detailed below) in testing the "stand-alone" version, the process passes the library validation (and does not pass when less entitlements are used).

[less entitlements]

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.security.app-sandbox</key>
	<true/>
	<key>com.apple.security.cs.disable-library-validation</key>
	<true/>
	<key>com.apple.security.files.bookmarks.app-scope</key>
	<true/>
	<key>com.apple.security.files.user-selected.read-write</key>
	<true/>
	<key>com.apple.security.network.client</key>
	<true/>
</dict>
</plist>

[more entitlements]

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.security.app-sandbox</key>
	<true/>
	<key>com.apple.security.cs.allow-dyld-environment-variables</key>
	<true/>
	<key>com.apple.security.cs.allow-jit</key>
	<true/>
	<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
	<true/>
	<key>com.apple.security.cs.disable-library-validation</key>
	<true/>
	<key>com.apple.security.files.bookmarks.app-scope</key>
	<true/>
	<key>com.apple.security.files.user-selected.read-write</key>
	<true/>
	<key>com.apple.security.network.client</key>
	<true/>
</dict>
</plist>

I haven't yet isolated which entitlement/s enable the validation to pass (Doing so is currently lower priority)

But to confirm, using [more entitlements] didn't work when attempting to inherit them from the parent app (The same issue arises.), hence the apparent need for the another approach like 'CFMessagePort'. If that approach doesn't work for the purpose of avoiding having to inherit, it would be key to understand why inheritance isn't working here.  

I plan to work toward that approach tomorrow

petiatil added the triage Please triage and relabel this issue label Oct 22, 2023

petiatil closed this as completed Oct 24, 2023

petiatil reopened this Oct 24, 2023

petiatil mentioned this issue Oct 25, 2023

Running whisperX in macOS / iOS sandbox m-bain/whisperX#149

Open

petiatil changed the title ~~Using codesigning flags still results in "no cdhash, completely unsigned?" errors (testing with --onedir executable)~~ Using codesigning flags still results in "no cdhash, completely unsigned?" errors (with --onedir executable or app bundle executable) Oct 29, 2023

petiatil mentioned this issue Nov 15, 2023

PyOpenSSL / CFFI freezes when running codesigned x64 binary on M1/Rosetta. #8099

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Using codesigning flags still results in "no cdhash, completely unsigned?" errors (with --onedir executable or app bundle executable) #8029

Using codesigning flags still results in "no cdhash, completely unsigned?" errors (with --onedir executable or app bundle executable) #8029

petiatil commented Oct 22, 2023 •

edited

rokm commented Oct 22, 2023

petiatil commented Oct 24, 2023 •

edited

rokm commented Oct 24, 2023

petiatil commented Oct 24, 2023 •

edited

petiatil commented Oct 24, 2023 •

edited

rokm commented Oct 24, 2023

petiatil commented Oct 25, 2023 •

edited

rokm commented Oct 25, 2023

rokm commented Oct 25, 2023

rokm commented Oct 25, 2023

rokm commented Oct 25, 2023 •

edited

rokm commented Oct 25, 2023 •

edited

petiatil commented Oct 26, 2023 •

edited

rokm commented Oct 26, 2023 •

edited

.DS_Store locations:

Some Python packages of the application:

petiatil commented Oct 27, 2023 •

edited

rokm commented Oct 28, 2023

petiatil commented Oct 29, 2023 •

edited

rokm commented Oct 30, 2023

petiatil commented Oct 31, 2023 •

edited

petiatil commented Oct 31, 2023 •

edited

rokm commented Oct 31, 2023

petiatil commented Nov 2, 2023 •

edited

petiatil commented Nov 6, 2023 •

edited

petiatil commented Nov 7, 2023 •

edited

petiatil commented Nov 9, 2023 •

edited

petiatil commented Nov 11, 2023 •

edited

rokm commented Nov 11, 2023 •

edited

petiatil commented Nov 12, 2023

rokm commented Nov 12, 2023

rokm commented Nov 13, 2023

petiatil commented Nov 14, 2023 •

edited

Using codesigning flags still results in "no cdhash, completely unsigned?" errors (with --onedir executable or app bundle executable) #8029

Using codesigning flags still results in "no cdhash, completely unsigned?" errors (with --onedir executable or app bundle executable) #8029

Comments

petiatil commented Oct 22, 2023 • edited

My use case:

Codesigning flags used:

Command structure (via Python script):

Entitlements to inherit:

Main app entitlements:

Error samples from the Console app:

What I've tried

What I'm considering

rokm commented Oct 22, 2023

petiatil commented Oct 24, 2023 • edited

rokm commented Oct 24, 2023

petiatil commented Oct 24, 2023 • edited

petiatil commented Oct 24, 2023 • edited

rokm commented Oct 24, 2023

petiatil commented Oct 25, 2023 • edited

rokm commented Oct 25, 2023

rokm commented Oct 25, 2023

rokm commented Oct 25, 2023

rokm commented Oct 25, 2023 • edited

rokm commented Oct 25, 2023 • edited

petiatil commented Oct 26, 2023 • edited

.DS_Store locations:

Python environment

Some Python packages of the application:

Other PyInstaller command details, if useful to know:

rokm commented Oct 26, 2023 • edited

.DS_Store locations:

Some Python packages of the application:

petiatil commented Oct 27, 2023 • edited

The temporary elements

My system and what I'm testing with

Next steps

rokm commented Oct 28, 2023

petiatil commented Oct 29, 2023 • edited

rokm commented Oct 30, 2023

petiatil commented Oct 31, 2023 • edited

petiatil commented Oct 31, 2023 • edited

rokm commented Oct 31, 2023

petiatil commented Nov 2, 2023 • edited

What else I'm considering

petiatil commented Nov 6, 2023 • edited

P2: Quick update:

petiatil commented Nov 7, 2023 • edited

P3: Update n2:

petiatil commented Nov 9, 2023 • edited

petiatil commented Nov 11, 2023 • edited

Update

rokm commented Nov 11, 2023 • edited

petiatil commented Nov 12, 2023

rokm commented Nov 12, 2023

rokm commented Nov 13, 2023

petiatil commented Nov 14, 2023 • edited

petiatil commented Oct 22, 2023 •

edited

petiatil commented Oct 24, 2023 •

edited

petiatil commented Oct 24, 2023 •

edited

petiatil commented Oct 24, 2023 •

edited

petiatil commented Oct 25, 2023 •

edited

rokm commented Oct 25, 2023 •

edited

rokm commented Oct 25, 2023 •

edited

petiatil commented Oct 26, 2023 •

edited

rokm commented Oct 26, 2023 •

edited

petiatil commented Oct 27, 2023 •

edited

petiatil commented Oct 29, 2023 •

edited

petiatil commented Oct 31, 2023 •

edited

petiatil commented Oct 31, 2023 •

edited

petiatil commented Nov 2, 2023 •

edited

petiatil commented Nov 6, 2023 •

edited

petiatil commented Nov 7, 2023 •

edited

petiatil commented Nov 9, 2023 •

edited

petiatil commented Nov 11, 2023 •

edited

rokm commented Nov 11, 2023 •

edited

petiatil commented Nov 14, 2023 •

edited