Debugpy cannot connect to process using process ID on Ubuntu 22.10

[Joebeazelman]

I'm encountering an issue where debugpy is unable to connect to another application running embedded Python. When I choose the process ID, the debug window keeps repeating that it's attaching, but eventually times out. The Debug window suggests setting the PYDEVD_GDB_SCAN_SHARED_LIBRARIES.

I tried to configure VS Code using a port number and server address, but VS Code reports a connection refused error. I thought the port was blocked, but I am running on a VM without a firewall. I've tried every possible solution, including deleting and reinstalling VS Code. I even went as far as setting the environment variable export PYDEVD_GDB_SCAN_SHARED_LIBRARIES="libdl, libltdl, libc, libfreebl3", but it still cannot connect.

I've attached the log files with this issue.

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Attach using Process Id",
            "type": "python",
            "request": "attach",
            "processId": "${command:pickProcess}",
            "justMyCode": true,
            "logToFile": true
        }
    ]
}

Environment data

• debugpy version: 1.6.6 (1.6.5 target)
• OS and version: Ubuntu 22.10
• Python version (& distribution if applicable, e.g. Anaconda): 3.10.7
• Using VS Code or Visual Studio: 1.74.3

Actual behavior

Error messages from embedded python

ConnectionRefusedError: [Errno 111] Connection refused
Traceback (most recent call last):
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/adapter/../../debugpy/../debugpy/server/attach_pid_injected.py", line 77, in attach
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/public_api.py", line 31, in wrapper
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/server/api.py", line 143, in debug
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/server/api.py", line 141, in debug
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/server/api.py", line 297, in connect
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/server/api.py", line 45, in _settrace
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2821, in settrace
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2902, in _locked_settrace
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 1421, in connect
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 493, in start_client
ConnectionRefusedError: [Errno 111] Connection refused
Traceback (most recent call last):
File "", line 1, in
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/adapter/../../debugpy/../debugpy/server/attach_pid_injected.py", line 90, in attach
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/adapter/../../debugpy/../debugpy/server/attach_pid_injected.py", line 77, in attach
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/public_api.py", line 31, in wrapper
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/server/api.py", line 143, in debug
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/server/api.py", line 141, in debug
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/server/api.py", line 297, in connect
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/server/api.py", line 45, in _settrace
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2821, in settrace
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2902, in _locked_settrace
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 1421, in connect
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.1.10091012/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 493, in start_client
ConnectionRefusedError: [Errno 111] Connection refused
import debugpy; print(debugpy.version)
1.6.5

debugpy.zip

[int19h]

Attach-to-PID ultimately just injects code into the process that sets up a server, so if regular attach with ports is not working because connections get rejected, I would expect this to be similar. Judging by the stack trace, the code was indeed injected correctly, and also fails because of the connection issue.

Can you clarify the arrangement for either scenario here? For attach-to-PID, is everything (including VSCode) running inside the VM? And for attach-to-port, was there two different machines involved, or is it also all on the VM, connecting localhost-to-localhost? Also, it would be interesting to see the logs from when the latter fails.

[Joebeazelman]

Everything is running on the same VM: the target app, script, debugger and VSCode. The script being debugged is executed by the target application running embedded Python. I'm confused as to the purpose of specifying the port and server address. I thought it is only necessary for remote debugging.

I'm merely following the online examples since I can't get my PID-only settings to work. On MacOS Monterey, I can debug the same application without specifying the server and port. I went ahead and configured it anyway on Ubuntu, figuring Linux is a more primitive OS written by a guy who lived with his parents and boasts about hand assembling x86 instructions. It doesn't help that the OS freaks out without provocation.

[int19h]

No worries, I was confused by the bit about "tried to configure VS Code using a port number and server address" - I thought you had it set up that way from both ends, and it worked. If you use attach-to-PID, host & port shouldn't be specified, since they're automatically configured.

If you're having problem with attach-to-PID on Linux specifically, it's likely because of this setting - it needs to be set to 0 to work:
https://linux-audit.com/protect-ptrace-processes-kernel-yama-ptrace_scope/

[Joebeazelman]

I believe I already tried this option (see ptrace setting below). I also made sure I didn't set the port or server address. I am still getting the sam exact error message.

 joebeazelman@joebeazelman-VirtualBox:~$ sysctl kernel.yama.ptrace_scope
 kernel.yama.ptrace_scope = 0

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Attach using Process Id",
            "type": "python",
            "request": "attach",
            "processId": "${command:pickProcess}",
            "justMyCode": true,
            "logToFile": true
        }
    ]
}

debugger.vscode_7e3774a2-1ba2-4114-95c6-b6d028e1ef45.log

[int19h]

That log won't give us much, unfortunately, because it's from the client (VSCode) side. Since this is attach-by-PID, and thus you can't use the debugpy API or CLI to tell the server to log when it starts, you'll need to use an environment variable for this when spawning the process you're trying to attach to: DEBUGPY_LOG_DIR=..., pointing it at the directory where the logs should be dumped.

[Joebeazelman]

OK. I set the environment variable and pointed it to my logs folder. Please find the enclosed logs.

Just for your information, the example for executing the debugger via the command line creates a folder with the name "~" with a subfolder named "logs" (see https://github.com/microsoft/debugpy/wiki/Enable-debugger-logs at the bottom). It doesn't correctly expand the tilde character into the user's home folder. Why doesn't UNIX finally die a dignified death so we can have usable interfaces? I can't believe we're still using such a bad user interface from the days when computers had limited resources. Surprisingly, Multics, a predecessor of UNIX, had a far superior command-line.

debugpy.pydevd.9579.log
debugpy.server-9579.log

[int19h]

Expansion of ~ is supposed to be something that (Bourne-compatible) shell does on Unix rather than individual tools, so it only applies when you set the variable via such a shell; we need to make this clearer in the docs.

Are you sure that the environment variable is in the effect for the process being attached to? This pair of logs is from the debugpy process that is spawned by VSCode to perform the injection, which seems to succeed. But there should be another pair generated from the process as soon as injected code begins to run. It might be that the failure happens when it's trying to create the log, but let's make sure that it's not something simple first - can you print out os.environ in the target process before trying to attach?

[Joebeazelman]

Fortunately, the host application provides a Python interpreter console, so I was able to retrieve os.envion as shown below.

import os
print(os.environ)
environ({'SHELL': '/bin/bash', 'SESSION_MANAGER': 'local/joebeazelman-VirtualBox:@/tmp/.ICE-unix/1239,unix/joebeazelman-VirtualBox:/tmp/.ICE-unix/1239', 'QT_ACCESSIBILITY': '1', 'COLORTERM': 'truecolor', 'XDG_CONFIG_DIRS': '/etc/xdg/xdg-ubuntu:/etc/xdg', 'SSH_AGENT_LAUNCHER': 'openssh', 'XDG_MENU_PREFIX': 'gnome-', 'GPS_STARTUP_XDG_DATA_DIRS': '/usr/share/ubuntu:/usr/local/share/:/usr/share/:/var/lib/snapd/desktop', 'GNOME_DESKTOP_SESSION_ID': 'this-is-deprecated', 'GPS_STARTUP_CHARSET': '', 'CHARSET': 'ISO-8859-1', 'GNOME_SHELL_SESSION_MODE': 'ubuntu', 'SSH_AUTH_SOCK': '/run/user/1000/keyring/ssh', 'XDG_CONFIG_HOME': '/opt/gnatstudio/etc', 'XMODIFIERS': '@im=ibus', 'DESKTOP_SESSION': 'ubuntu', 'GDK_PIXBUF_MODULE_FILE': '/opt/gnatstudio/lib/gnatstudio/gdk-pixbuf-2.0/2.10.0//loaders.cache', 'GTK_DATA_PREFIX': '/opt/gnatstudio', 'GTK_MODULES': 'gail:atk-bridge', 'PWD': '/home/joebeazelman', 'XDG_SESSION_DESKTOP': 'ubuntu', 'LOGNAME': 'joebeazelman', 'GTK_EXE_PREFIX': '/opt/gnatstudio', 'XDG_SESSION_TYPE': 'wayland', 'SYSTEMD_EXEC_PID': '1257', 'XAUTHORITY': '/run/user/1000/.mutter-Xwaylandauth.HOZA01', 'GI_TYPELIB_PATH': '/opt/gnatstudio/lib/girepository-1.0', 'HOME': '/home/joebeazelman', 'USERNAME': 'joebeazelman', 'IM_CONFIG_PHASE': '1', 'LANG': 'en_US.UTF-8', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.webp=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:.xspf=00;36:', 'XDG_CURRENT_DESKTOP': 'ubuntu:GNOME', 'VTE_VERSION': '7000', 'WAYLAND_DISPLAY': 'wayland-0', 'GPS_STARTUP_GTK_DATA_PREFIX': '', 'GNOME_TERMINAL_SCREEN': '/org/gnome/Terminal/screen/0540cf69_0a12_4ce9_8c7f_400c226421c5', 'GPS_STARTUP_GTK_EXE_PREFIX': '', 'DYLD_FALLBACK_LIBRARY_PATH': '', 'GNOME_SETUP_DISPLAY': ':1', 'GPS_STARTUP_GTK2_RC_FILES': '', 'GPS_STARTUP_LD_LIBRARY_PATH': '', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'XDG_SESSION_CLASS': 'user', 'TERM': 'dumb', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'USER': 'joebeazelman', 'GNOME_TERMINAL_SERVICE': ':1.114', 'GPS_STARTUP_GDK_PIXBUF_MODULE_FILE': '', 'GPS_STARTUP_DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus', 'DISPLAY': ':0', 'SHLVL': '1', 'GDK_PIXBUF_MODULEDIR': '/opt/gnatstudio/lib/gnatstudio/gdk-pixbuf-2.0/2.10.0//loaders', 'QT_IM_MODULE': 'ibus', 'GPS_STARTUP_FONTCONFIG_FILE': '', 'LD_LIBRARY_PATH': '', 'GPS_STARTUP_GSETTINGS_BACKEND': '', 'XDG_RUNTIME_DIR': '/run/user/1000', 'GSETTINGS_BACKEND': 'memory', 'GPS_STARTUP_GI_TYPELIB_PATH': '', 'GPS_STARTUP_DYLD_FALLBACK_LIBRARY_PATH': '', 'DEBUGINFOD_URLS': 'https://debuginfod.ubuntu.com ', 'FONTCONFIG_FILE': '/opt/gnatstudio/etc/fonts/fonts.conf', 'GPS_STARTUP_XDG_CONFIG_HOME': '', 'XDG_DATA_DIRS': '/opt/gnatstudio/share', 'GPS_STARTUP_GDK_PIXBUF_MODULEDIR': '', 'PATH': '/opt/Alire/bin/:/opt/gnatstudio/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/opt/gnatstudio/bin:/opt/gnatstudio/bin', 'GDMSESSION': 'ubuntu', 'DBUS_SESSION_BUS_ADDRESS': 'null', 'GTK_OVERLAY_SCROLLING': '0', 'PYTHONPATH': '/opt/gnatstudio/share/gnatstudio/python', 'PYDEVD_USE_FRAME_EVAL': 'NO'})

On a whim, I experimented by importing my script directly and I got the following error:

import call_me_alire
Traceback (most recent call last):
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.2.0/pythonFiles/lib/python/debugpy/server/api.py", line 220, in listen
_adapter_process = subprocess.Popen(
File "/opt/gnatstudio/share/gnatstudio/python/lib/python3.9/subprocess.py", line 951, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/opt/gnatstudio/share/gnatstudio/python/lib/python3.9/subprocess.py", line 1821, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: ''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/home/joebeazelman/.gnatstudio/plug-ins/call_me_alire.py", line 13, in
debugpy.listen(5678)
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.2.0/pythonFiles/lib/python/debugpy/public_api.py", line 31, in wrapper
return wrapped(*args, **kwargs)
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.2.0/pythonFiles/lib/python/debugpy/server/api.py", line 143, in debug
log.reraise_exception("{0}() failed:", func.name, level="info")
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.2.0/pythonFiles/lib/python/debugpy/server/api.py", line 141, in debug
return func(address, settrace_kwargs, **kwargs)
File "/home/joebeazelman/.vscode/extensions/ms-python.python-2023.2.0/pythonFiles/lib/python/debugpy/server/api.py", line 234, in listen
raise RuntimeError("error spawning debug adapter: " + str(exc))
RuntimeError: error spawning debug adapter: [Errno 13] Permission denied: ''

This is probably the reason why it can't attach to the process. Why didn't this show up in the logs or VSCode's console? This is classic security through obfuscation, creating the worst security where users authorize excessive privileges or turn off security entirely. Unfortunately, I'm no UNIX expert and don't know how to resolve this issue.

---

I thought I would also mention some oddities that may or may not be relevant to the debugging issue. First, the Python version reported by VSCode is version 3.10.7, whereas the host application's version is 3.9.9:

print(sys.version)
3.9.9 (main, Mar 22 2022, 10:26:16)
[GCC 8.3.1 20190528 (for GNAT Pro 20.0w 20190528)]

Does this cause any issues when debugging? Second, there's a pesky error claiming Python cannot find one of the modules provided by the host.

Import "GPS" could not be resolved

I get the same error on the Mac, but once the debugging starts, the error goes away. Both of these issues also occur on the Mac, but debugging still works. It's unclear which interpreter does the interpreting when the application is attached. It's all intergalactic voodoo to me.

[int19h]

Python version disparity is okay so long as they're all supported versions (3.7+).

The error couldn't be displayed in VSCode because it happens before the connection between debuggee and VSCode is established - the "adapter process" that it was trying to spawn but failed is what VSCode would normally connect to. However, the error should be displayed in the console - stderr, specifically - if the debuggee has one. Could it be that the host app redirects output somewhere, e.g. an internal output window?

Looking at the env var dump, I don't see DEBUGPY_LOG_DIR in there, so it doesn't look like it gets propagated from wherever it was set to the GNAT Studio process. Can you try setting it directly in the terminal from which you launch Studio?

Regarding the error message that you've got when trying to run your script directly, did you use --listen (or debugpy.listen) for that? It should work, but if you use VSCode to perform attach-to-PID via "processId", it uses --connect instead, which does not require any helper processes to be spawned on debugpy side, so it's not quite the same thing. The error that you're getting when trying it manually is because the server needs to be able to spawn the adapter process for --listen, and it does so using sys.executable as the Python interpreter. In embedded scenarios, the latter usually points to the binary of the host app or is unset, so you need to explicitly tell it which interpreter to use by using --configure-python ... or debugpy.configure(python='...') - in the latter case, it must be done before you call listen(). In general, connect() is simpler and with fewer moving parts on debuggee side, which is why we use it by default for the PID scenario.

[int19h]

Oh, and regarding "unresolved imports" - this is coming from the editor, not at runtime, right? The editor needs to be aware where all those custom modules are located so that it can analyze them. If GNAT Studio ships with a full-fledged Python distribution (which includes the python binary), it should be possible to manually register it as a custom Python environment. Otherwise, you should use "python.analysis.extraPaths" in settings.json to make it aware of those modules.

[Joebeazelman]

OK. I've attached the log files. I am nearing the point of calling this exercise a failure. The problem stems from trying to teach an old dog (UNIX), well over his dog years (born in 1969), new tricks (modern software). The logs don't pinpoint exactly what the problem is. You're correct, GnatStudio was consuming the log messages and outputting them to its python console window. By the way, you seem to know something about GnatStudio? Have you worked with it in the past?

debugpy.pydevd.4248.log
debugpy.server-4248.log

[int19h]

I'm familiar with Ada, and I've played with GNAT Studio before in that context. But unfortunately, I know nothing about its Python extensibility story, so this is all just guesswork based on how apps that embed Python usually do it.

And yes, it looks like the logs don't say much even when pieced together. The adapter is opening a socket for listen, and that much succeeds, but it never gets the incoming connection. And at the same time, on the other end, the code that was injected into Studio runs and tries to connect, but connection is rejected. This usually indicates some kind of networking issue, or a highly unusual setup where e.g. 127.0.0.1 is not a loopback interface, or ephemeral ports are off-limits for listening.

If you want to try chasing this down, I think it would be best to remove the debuggee from the picture and see if you can connect to VSCode (running in "listen": {"port": ...} ) using telnet or netcat or something like that. It might be able to produce better error messages than Python does, and if not, at least reduce the scope in which to look for possible problems. Or
see if you can get a basic TCP/IP client and server sample connect on that machine via localhost.

[github-actions]

This issue was closed because it has been stalled for 30 days with no activity. If the issue still persists, please reopen with the information requested. Thanks.