Docs/howto/remote_debugging: Give non-sudo suggestions first

[geofft]

sudo is much too powerful and unnecessary for debugging your own processes. Start with guidance on how to opt into being debugged without the use of sudo, and clarify that sudo and equivalent options like CAP_SYS_PTRACE are giant hammers, but leave the sudo option documented in case you're having trouble getting something else working.

---

📚 Documentation preview 📚: https://cpython-previews--139139.org.readthedocs.build/

[geofft]

cc @ivonastojanovic, @pablogsal

Rendered

Let me know if you think this is nontrivial enough to have opened an issue first for discussion - though my issue would probably have contained the actual proposed new text here.

I've manually tested the instructions here with python-build-standalone's builds on Linux and macOS. The macOS builds are non-framework, hence the caveat. I can find a framework build and add that to the body if you think that's needed in this PR.

I haven't tested on Windows, but I suspect that admin access isn't really needed there either. I can come back and fix that up.

See https://jvns.ca/blog/2020/04/29/why-strace-doesnt-work-in-docker/ for why SYS_PTRACE is incorrect on Docker and no longer needed - though given that the argument against it is from 2020 and the fixed version of Docker is from 2019, maybe you ran into it more recently and I'm missing something?

Also, because the error messages in Lib/asyncio/tools.py and Lib/pdb.py refer to versioned copies of this docs, I would really really like this on the 3.14 branch as well, though I understand if that's hard at this point.

[JelleZijlstra]

Also, because the error messages in Lib/asyncio/tools.py and Lib/pdb.py refer to versioned copies of this docs, I would really really like this on the 3.14 branch as well, though I understand if that's hard at this point.

Docs improvements are generally fine even in the RC phase. Definitely it should be fine to backport after 3.14.0 is out; it's up to Hugo whether to merge a backport before that.

[pablogsal]

I think this is a good improvement and I like the idea. The only thing is that I am concerned about the macOS case as the non-sudo approach is quite complex and most beginners are not going to be able to use it and offers a bad experience

[pablogsal]

This needs another sentence explaining how to work around it (add the CAP_SYS_PTRACE capability or other similar options). Currently this is too far from where we mention it and it doesn't appear directly

[geofft]

If the system call is disabled because of a filter, CAP_SYS_PTRACE doesn't help you. The filtering happens at system call entry well before permissions are checked.

It happens to be the case that if you add CAP_SYS_PTRACE, Docker and code derived from it (e.g. containerd) will change their system call filter to no longer exclude ptrace. See the end of the jvns blog post I linked above. However, as the blog post mentions, it is also the case that current versions of this code unconditionally permit ptrace on kernel 4.8 and above. See moby/moby@1124543ca8 (linked in jvns' post), released in Docker 19.03 (July 2019), and containerd/containerd@94faa70df4, released in containerd 1.6.7 (August 2022). So a much better solution, if it is possible, is to upgrade to one of these versions.

I didn't realize the containerd change was so much more recent than the Docker change, so I think it's plausible that people are still on that version. In that case I think it's better to pass --security-opt seccomp=unconfined than --cap-add=SYS_PTRACE. Technically both do more than just permitting the one syscall, the former turns off the filter entirely (i.e. it increases the attack surface of the host kernel presented to the container), but I think that's safer. I'll add that in.

[geofft]

The only thing is that I am concerned about the macOS case as the non-sudo approach is quite complex and most beginners are not going to be able to use it and offers a bad experience

What are you concerned about specifically? (To be clear, I agree with this sentiment overall, I'm just trying to think through whether there's something we can do about it.)

We could ship a small bit of code that does this re-signing for you, e.g. a shell script somewhere in the CPython distribution, or something in the standard library e.g. pdb.enable_macos_debugging(interpreter=sys.executable).

We could consider adding this entitlement to the prebuilt macOS binaries. It does reduce the security posture but I sort of suspect that isn't meaningful for something like Python that runs arbitrary code anyway. I am slightly tempted to do it for python-build-standalone's prebuilt binaries or to add a flag to uv to re-sign the binary with this entitlement. (Since it's an ad-hoc entitlement we can do this either at build time without a cert or at install time.)

We could add a ./configure option to add the entitlement automatically at build time, or roll it into --with-pydebug. I would almost certainly be comfortable adding this entitlement by default to python-build-standalone's prebuilt debug binaries, though that's only helpful if we're comfortable advertising those binaries on this page.

This of course assumes you have write access to the Python binary. I suspect that getting this to work with Apple's own builds of Python, once they pick up 3.14, is going to be hard, and I don't have a great answer there (though I am of course a proponent of getting your Python in some other way....). One option is to add a wrapper for creating a venv with --copies and modifying the copy in bin/, though note that Apple patches out support for --copies, and that functionality is currently broken on python-build-standalone for related reasons, which is on my todo list to fix.

Are you concerned about the "if it's a framework build" part? I can figure out exact instructions for that, and that's a little bit more of an argument for shipping some code that automatically does the right thing.