Debugging with LLDB doesn't work for universal2 installer on macOS Big Sur

[ronaldoussoren]

BPO	44234
Nosy	@ronaldoussoren, @ned-deily

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-05-25.14:20:50.394>
labels = ['OS-mac', 'type-bug', '3.9', 'build']
title = "Debugging with LLDB doesn't work for universal2 installer on macOS Big Sur"
updated_at = <Date 2021-05-26.14:53:38.270>
user = 'https://github.com/ronaldoussoren'

bugs.python.org fields:

activity = <Date 2021-05-26.14:53:38.270>
actor = 'ronaldoussoren'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build', 'macOS']
creation = <Date 2021-05-25.14:20:50.394>
creator = 'ronaldoussoren'
dependencies = []
files = []
hgrepos = []
issue_num = 44234
keywords = []
message_count = 3.0
messages = ['394356', '394417', '394441']
nosy_count = 2.0
nosy_names = ['ronaldoussoren', 'ned.deily']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44234'
versions = ['Python 3.9']

[ronaldoussoren]

I've installed python 3.9.5 using the universal2 installer on my M1 MacBook.

Debugging this python using lldb doesn't work because the debugger cannot attach to the (newly launched) binary. This appears to be due to a missing entitlement in the signed executable.

Background: https://developer.apple.com/forums/thread/676028?answerId=666834022#666834022 (in particular the first reply).

[ned-deily]

There's a reason for that. What the discussion in that link fails to note is that elsewhere Apple warns that that entitlement is understandably prohibited in distributions submitted to the Apple notarization service. See the "Avoid the Get-Task-Allow Entitlement" section here: https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues. I haven't verified that that is still the case but I'd be surprised if it weren't.

I think one solution (and possibly the best) might be to manually re-codesign the installed binaries if you need to use lldb with the python.org binaries. If someone comes up with something that works, we could document it somewhere with appropriate warnings about security risks and advice to reinstall when finished.

[ronaldoussoren]

Should we add documentation about this?

The truly annoying bit about that discussion is that its probably easier to just install gdb and use that to debug extensions.

[artemmukhin]

I have found the following workaround.

1. Create a certificate (can be skipped if you already have an Apple Developer ID certificate)
   1.1. Open Keychain Access.
   1.2. Choose Keychain Access > Certificate Assistant > Create Certificate...
   1.3. Choose a name <MyName>
   1.4. Set Certificate Type to Code Signing

2. Create an empty file my_entitlements.xml

3. Run $ codesign -d -vvv --entitlements :- /Library/Frameworks/Python.framework/Versions/<version>/Resources/Python.app and copy XML output to my_entitlements.xml.

After step 3, my_entitlements.xml may look like this

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
        "https://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>com.apple.security.automation.apple-events</key>
        <true/>
        <key>com.apple.security.cs.allow-dyld-environment-variables</key>
        <true/>
        <key>com.apple.security.cs.disable-executable-page-protection</key>
        <true/>
        <key>com.apple.security.cs.disable-library-validation</key>
        <true/>
    </dict>
</plist>

4. Add <key>com.apple.security.get-task-allow</key><true/> to my_entitlements.xml.

After step 4, my_entitlements.xml may look like this

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
        "https://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>com.apple.security.automation.apple-events</key>
        <true/>
        <key>com.apple.security.cs.allow-dyld-environment-variables</key>
        <true/>
        <key>com.apple.security.cs.disable-executable-page-protection</key>
        <true/>
        <key>com.apple.security.cs.disable-library-validation</key>
        <true/>
        <key>com.apple.security.get-task-allow</key>
        <true/>
    </dict>
</plist>

5. Run $ codesign --remove-signature /Library/Frameworks/Python.framework/Versions/<version>/Resources/Python.app

6. Run $ codesign -s <MyName> --entitlements my_entitlements.xml --deep /Library/Frameworks/Python.framework/Versions/<version>/Resources/Python.app

@ronaldoussoren @ned-deily Could you please verify if this can be used as an unsafe solution?

[artemmukhin]

In addition, I would like to provide more details on why this issue is important.

Apart from the extensions debugging mentioned above, this issue is critical for the Attach to Process functionality provided by IDEs based on PyDev Debugger, particularly Visual Studio Code and PyCharm. PyDev Debugger relies on LLDB to attach to a running Python process. So if the process runs the official Python package from www.python.org, PyDev Debugger cannot attach. This problem is described in fabioz/PyDev.Debugger#234; it mentions Python 3.11, but I am facing the issue on Python 3.9 and Python 3.10 as well. And of course, the above-mentioned debugging of extension modules is one of the significant features of Python IDEs too.

Currently, the workaround for macOS users, besides the thorny way of signing Python.app manually, is to use another Python distribution that does not use hardened runtime, such as Python provided via Homebrew.

Speaking of other development tools that have also been impacted by missing com.apple.security.get-task-allow, I have found this discussion regarding the R project. As I can see, the final decision was to include the com.apple.security.get-task-allow entitlement.

I noticed in #84379 that the idea is to keep the entitlements list as short as possible, and I completely understand the reason behind that. However, considering that this particular issue is critical for Python tooling, I would like to highlight this and request the maintainers' opinions.

Cc @ned-deily

[ronaldoussoren]

I have found the following workaround.

[...]

@ronaldoussoren @ned-deily Could you please verify if this can be used as an unsafe solution?

An easier solution is to drop steps 1 to 4 and replace step 6 by

$ codesign -s - -deep /Library/Frameworks/Python.framework/Versions/<version>/Resources/Python.app

On an Intel Mac you can leave of step 6. This is basically what I do on my M1 laptop when I need to debug, although I also do this to /Library/Frameworks/Python.framework/Versions/<version>/bin/python<version>.

You loose the hardened runtime that way, and maybe some access to more sensitive system APIs, but neither of them should affect debugging for more code.

Note that we don't have a choice w.r.t. the hardened runtime: If we want to distribute a signed installer we have to use the hardened runtime. I'd be -1 on starting to ship a non-signed installer, for various reasons.

Adding com.apple.security.get-task-allow would IMHO be acceptable, but I don't know if this entitlement can be used with Developer Certificates: Apple's documentation only mentions the entitlement in passing in the "Debugging Tool" entitlement and does not document it separately.

[ronaldoussoren]

@ned-deily : Get Task Allow should be OK according to the page you mentioned earlier, it says:

To enable debugging a plug-in in the context of a host executable, the host can include the com.apple.security.get-task-allow entitlement if it also includes the Disable Library Validation Entitlement. Don’t disable library validation for executables that don’t host plug-ins because library validation protects them from loading untrusted code.

We ship with the com.apple.security.cs.disable-library-validation entitlement to be able to load unsigned extension modules, and the security risk of injecting code in binaries should be fairly low as users could more easily just execute the python code they want to run in the first place.

[ned-deily]

Thanks, @ronaldoussoren. I'll test packaging with that.

[artemmukhin]

@ronaldoussoren Thank you for describing a more straightforward workaround for the issue.

@ned-deily I greatly appreciate your attention to this issue. Please let me know if there is anything I can do to help move this forward. For example, I would be happy to verify an experimental package build for debugging purposes.

[ned-deily]

Sorry this slipped under the radar! As of Python 3.14.0b1, the python.org macOS binaries are now signed with the com.apple.security.cs.disable-library-validation as suggested. It would be great to have confirmation that this does solve the debugging with LLDB issue for you.

[ronaldoussoren]

Sorry this slipped under the radar! As of Python 3.14.0b1, the python.org macOS binaries are now signed with the com.apple.security.cs.disable-library-validation as suggested. It would be great to have confirmation that this does solve the debugging with LLDB issue for you.

It works for me, in a fresh install of 3.14.0b1 using the python.org installer I can use LLDB to debug the python interpreter.

Thanks!