[CDT] gdbserver not found #36

mcoufal · 2021-06-02T13:27:56Z

I can't make debugger to work, I always end up with an error that command 'gdbserver' is not found.

How to reproduce:

create simple Hello World C++ Project with Linux GCC toolchain
build project
set a line breakpoint somewhere in main
debug as Local C/C++ Application

Console view now shows an error: '/bin/sh: gdbserver: command not found'.
Might be connected to [https://github.com//issues/35](issue 35).

My configuration:
Eclipse SDK
Version: 2021-06 (4.20)
Build id: I20210601-0600
Java runtime version: 15.0.2+7-27
Eclipse C/C++ Development Tools 10.3.0.202105311920

A host `/usr/bin` folder mounted to `/var/run/host/usr/bin` is added to the `$PATH` in order to enable access to the programs installed on the host and not existing on the sandbox, for example, such as `gdbserver` Issue: flathub#36 Signed-off-by: Victor Rubezhny <vrubezhny@redhat.com>

vrubezhny · 2021-07-08T12:46:16Z

When using a build made of PR #45 I see that gdbserver executable now can be found, but process still cannot be normally started.

I found that the command line that is used to launch gdbserver does look like:

flatpak-spawn -v --host --watch-bus /bin/sh -c gdbserver :2345 /home/user/projects/workspace/cdtCppTest/Debug/cdtCppTest

when starting a debugger ( Debug as -> C/C++ Application) it tries to execute the command line above and finally fails with a TimeoutException:

org.eclipse.core.runtime.CoreException: Timed out trying to launch GDB.
	at org.eclipse.cdt.dsf.concurrent.Sequence.abortExecution(Sequence.java:599)
	at org.eclipse.cdt.dsf.concurrent.Sequence$2.handleErrorOrWarning(Sequence.java:448)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.handleFailure(RequestMonitor.java:433)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.handleCompleted(RequestMonitor.java:393)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor$2.run(RequestMonitor.java:317)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:831)

Normally, when a gdbserver process is started it outputs a text like the following:

Process /home/user/projects/workspace/cdtCppTest/Debug/cdtCppTest created; pid = 999092
Listening on port 2345

So I see a text like this in the console, but then it looks like nothing more is being communicated between server process and CDT GDBBackend. At least it looks like this.

vrubezhny · 2021-07-08T13:59:49Z

Probably, the real reason is in flatpak-spawn'ed gdb process:

At startup, GDB reads the following init files and executes their commands:
   * system-wide init files: /etc/gdbinit

The host /etc directory is not available inside a sandbox (it has its own '/etc' directory, so flatpak-spawn'ed gdb process has no access to host's /etc/gdbinit file.

vrubezhny · 2021-07-09T16:02:04Z

@jjohnstn @akurtakov
The problem with Flatpak CDT Debug is somewhere in flatpak-spawn'ed-GDB-to-GDBBackend communication... GDB id flatpak-spawned process with --interpreter mi2 argument and -ex "new-ui mi /dev/pts/1" command sent to it. It's started like this:

flatpak-spawn --verbose  --host --watch-bus  gdb --interpreter mi2  --nx  -q  --interpreter console -ex "new-ui mi /dev/pts/1" -ex "set pagination off" -ex "show version"

"/dev/pts/1" here is a PTY name that GDBBackend(GDBBackend_7_12) is created to listen for mi2 interpreter output and [probably] to send commands to.

When debug starts, GDBBackend tries to read all lines from "/dev/pts/1" until it received (dgb) prompt. And here it get stuck on inputReader.readLine() - it never receives any lines (even chars) from this stream.

When I manually start the similar commands (but replacing "/dev/pts/1" by a real terminal name), like:

In terminal 1 I execute tty command to get know the terminal name:

$  tty
/dev/pts/14

then in terminal 2 I execute (putting the /dev/pts/14 name received at the previous step):

flatpak-spawn --verbose  --host --watch-bus  gdb --interpreter mi2  --nx  -q  --interpreter console -ex "new-ui mi /dev/pts/14" -ex "set pagination off" -ex "show version"

In terminal 1 I start receiving the GDB prompt and the mi2 interpreter echo/log of the commands I run from the terminal 2, like:

 =thread-group-added,id="i1"
(gdb) 
=cmd-param-changed,param="pagination",value="off"
=cmd-param-changed,param="pagination",value="on"
=cmd-param-changed,param="pagination",value="off"

So, it looks like mi2 interpreter's output really exists (and I can read it by a terminal that I specified as the command argument).

But when I start CDT debug - mi2 interpreter output is never received... Everything gets stuck on inputReader.readLine() at: https://git.eclipse.org/c/cdt/org.eclipse.cdt.git/tree/dsf-gdb/org.eclipse.cdt.dsf.gdb/src/org/eclipse/cdt/dsf/gdb/service/GDBBackend.java?id=05c45c0bae9c85b917309712306aadee052f3d8f#n568

I tried to read flatpak-spawn'ed GDB process output - and it receives all the DGB console output (including the required prompt (gdb) but,off course, not in mi2 interpreter's format (so it cannot be used further in the debug session).

@jjohnstn @akurtakov Any thoughts on how this can be fixed? Maybe we should fix some terminal setting when we create it or I don't know.. some flatpak-spawn argument is to be set to make it flash the streams (however I don't think it controls that stream anyhow)?

PS: In order to run Flatpak CDT debugger I've installed the build made of PR #45 (needed to make gdbserver available on sandbox) and then executed the following commands to link Host's directories on sandbox'es /etc directory (needed to make GDB init configurations available on sandbox) and to start org.eclipse.Java:

$ flatpak run --command=sh org.eclipse.Java
[📦 org.eclipse.Java ~]$ ln -s /var/run/host/etc/gdbinit /etc
[📦 org.eclipse.Java ~]$ ln -s /var/run/host/etc/gdbinit.d /etc
[📦 org.eclipse.Java ~]$ /app/bin/eclipse  &

vrubezhny · 2021-07-09T18:29:00Z

I'm not sure, maybe I did a wrong experiment, but IMHO the problem is that Flatpak CDT GDBBackend (running under the sandbox) tries to read a stream that is not sandbox'ed....

flatpak-spawn is a kind of portal that runs a not-sandbox'ed application and re-transmits its input/output/error streams to/from other sandbox'ed application. But all the other custom streams opened by such application, like the one opened directly from GDB for mi2 interpreter, are blocked and cannot reach sandbox'ed clients - they can be read/written only outside of sandbox (like I did in previous test with terminal).

To demo it I did almost the same steps I did in previous comment (#36 (comment)) but in first step I've used a terminal opened in flatpak:

In terminal 1 I executed tty command in sandbox' sh to get know the terminal name:

$ flatpak run --command=sh org.eclipse.Java
[📦 org.eclipse.Java ~]$ tty
/dev/console

then when in terminal 2 I executed (putting the /dev/console name received at the previous step):

flatpak-spawn --verbose  --host --watch-bus  gdb --interpreter mi2  --nx  -q  --interpreter console -ex "new-ui mi /dev/console" -ex "set pagination off" -ex "show version"

I did not received any (gdb) prompts nor any other output - looks exactly as I see it when I run CDT debug.

So, we need to find out a way to read/write a stream created by non-sandbox'ed application to/from a sandbox'ed one.

A host `/usr/bin` folder mounted to `/var/run/host/usr/bin` is added to the `$PATH` in order to enable access to the programs installed on the host and not existing on the sandbox, for example, such as `gdbserver`. PATH is set to include '/var/run/host/usr/bin/' 'gdbinit' and 'gdbinit.d' are mounted to '/etc' to be accessed by gdb '--device=all" permission is added to allow access to devices like /dev/pts/* from sandbox Issue: flathub#36 Signed-off-by: Victor Rubezhny <vrubezhny@redhat.com>

vrubezhny · 2021-07-09T23:11:06Z

There is some progress with build https://flathub.org/builds/#/builders/32/builds/53062, at least now we can start gdbserver and gdb and read the messages from gdb - not just hanging out on Mi-command reading until timeout happens.
But still it's not possible to debug due to the following error:

Error in final launch sequence:

Failed to execute MI command:
-target-select remote localhost:2345/tcp
Error message from debugger back end:
localhost:2345/tcp: No such file or directory.

jjohnstn · 2021-07-10T00:02:09Z

Hi Victor, I am running an old Flatpak 2020-09 SDK with CDT installed from the matching release repo and using my local gdb 7.6.1 and C/C++ debugging is working. The gdbserver gets kicked off and gdb appears to talk to it fine. There have been some mi issues reported lately in cdt-dev but they were for mac. Perhaps the problem is really tied to newest versions of gdb and/or CDT. My program is a simple hello world program that I created in my workspace but I also tested with a program that does a getchar() and that works as well.

…

-- Jeff J.

On Fri, Jul 9, 2021 at 12:02 PM Victor V. Rubezhny ***@***.***> wrote: @jjohnstn <https://github.com/jjohnstn> @akurtakov <https://github.com/akurtakov> The problem with Flatpak CDT Debug is somewhere in flatpak-spawn'ed-GDB-to-GDBBackend communication... GDB id flatpak-spawned process with --interpreter mi2 argument and -ex "new-ui mi /dev/pts/1" command sent to it. It's started like this: flatpak-spawn --verbose --host --watch-bus gdb --interpreter mi2 --nx -q --interpreter console -ex "new-ui mi /dev/pts/1" -ex "set pagination off" -ex "show version" "/dev/pts/1" here is a PTY name that GDBBackend(GDBBackend_7_12) is created to listen for mi2 interpreter output and [probably] to send commands to. When debug starts, GDBBackend tries to read all lines from "/dev/pts/1" until it received (dgb) prompt. And here it get stuck on inputReader.readLine() - it never receives any lines (even chars) from this stream. When I manually start the similar commands (but replacing "/dev/pts/1" by a real terminal name), like: - In terminal 1 I execute tty command to get know the terminal name: $ tty /dev/pts/14 - then in terminal 2 I execute (putting the /dev/pts/14 name received at the previous ster): flatpak-spawn --verbose --host --watch-bus gdb --interpreter mi2 --nx -q --interpreter console -ex "new-ui mi /dev/pts/14" -ex "set pagination off" -ex "show version" In terminal 1 I start receiving the GDB prompt and the mi2 interpreter echo/log of the commands I run from the terminal 2, like: =thread-group-added,id="i1" (gdb) =cmd-param-changed,param="pagination",value="off" =cmd-param-changed,param="pagination",value="on" =cmd-param-changed,param="pagination",value="off" So, it looks like mi2 interpreter's output really exists (and I can read it by a terminal that I specified as the command argument). But when I start CDT debug - mi2 interpreter output is never received... Everything gets stuck on inputReader.readLine() at: https://git.eclipse.org/c/cdt/org.eclipse.cdt.git/tree/dsf-gdb/org.eclipse.cdt.dsf.gdb/src/org/eclipse/cdt/dsf/gdb/service/GDBBackend.java?id=05c45c0bae9c85b917309712306aadee052f3d8f#n568 I tried to read flatpak-spawn'ed GDB process output - and it receives all the DGB console output (including the required prompt (gdb) but,off course, not in mi2 interpreter's format (so it cannot be used further in the debug session). Any thoughts on how this can be fixed? Maybe we should fix some terminal setting when we create it or I don't know.. some flatpak-spawn argument is to be set to make it flash the streams (however I don't think it controls that stream anyhow)? PS: In order to run Flatpak CDT debugger I've installed the build made of PR #45 <#45> (needed to make gdbserver available on sandbox) and then executed the following commands to link Host's directories on sandbox'es /etc directory (needed to make GDB init configurations available on sandbox) and to start org.eclipse.Java: $ flatpak run --command=sh org.eclipse.Java [📦 org.eclipse.Java ~]$ ln -s /var/run/host/etc/gdbinit /etc [📦 org.eclipse.Java ~]$ ln -s /var/run/host/etc/gdbinit.d /etc [📦 org.eclipse.Java ~]$ /app/bin/eclipse & — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJJC3P626VCOPJDMSSNATDTW4MQNANCNFSM456XX3BQ> .

vrubezhny · 2021-07-10T00:33:02Z

@jjohnstn Yes, off course, the newer versions could be a reason.
I'm currently on:

Flatpak 1.10.2
Flatpak Eclipse 2021-06 (4.20.0), Build id: 20210612-2011
GNU gdb/gdbserver (GDB) Fedora 10.1-4.fc33

Yet another problem s that (probably in case of problem on starting debugger) I have gdbserver running left in the system (until I restart eclipse or kill it), so every second debugger start ends with:

Can't bind address: Address already in use.
Exiting

Not sure if it needs to be loaded all the time (especially given that fact that it's given with the path to executable to debug at start) - but we're launching it on every debugger launch (and yes - that port number is blocked after the first attempt). But it probably another error.

jjohnstn · 2021-07-10T02:35:08Z

FWIW, multiple debugs work fine in my scenario as well. I can debug again and again and the gdbserver exits each time.

…

On Fri, Jul 9, 2021 at 8:33 PM Victor V. Rubezhny ***@***.***> wrote: @jjohnstn <https://github.com/jjohnstn> Yes, off course, the newer versions could be a reason. I'm currently on: - Flatpak Eclipse 2021-06 (4.20.0), Build id: 20210612-2011 - GNU gdb/gdbserver (GDB) Fedora 10.1-4.fc33 Yet another problem s that (probably in case of problem on starting debugger) I have gdbserver running left in the system (until I restart eclipse or kill it), so every second debugger start ends with: Can't bind address: Address already in use. Exiting Not sure if it needs to be loaded all the time (especially given that fact that it's given with the path to executable to debug at start) - but we're launching it on every debugger launch (and yes - that port number is blocked after the first attempt). But it probably another error. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJJC3JCTC4T2ZMUKGOMYULTW6IMVANCNFSM456XX3BQ> .

jjohnstn · 2021-07-14T20:44:22Z

After discussing with Jonah, the problem appears to be that CDT will use a full gdb console if the level of gdb supports it (which started in 7.12). There is a CDT call isFullGdbConsoleSupported() which when it returns true: there are two PTYs that connect GDB and Eclipse CDT, one for MI and one for CLI. When !“full gdb” only a single PTY connects GDB and CDT and all that communication is MI. The CLI in that case is wrapped in MI commands. It may be possible to patch the code to always return false in the case of flatpak. I will look into creating such a patch.

jjohnstn · 2021-07-15T20:13:34Z

The following bug has been opened and a gerrit has been submitted:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=574874

vrubezhny · 2021-08-03T14:01:34Z

@jjohnstn Thanks for the fixing the extra PTY for MI problem. It looks like it really helped to solve the block on reading the MI stream.
But now I have another problem while starting the debugger - it looks like gdb has a problem accessing localhost:2345/tcp specified in target-select command (or some other related problem):

!ENTRY org.eclipse.cdt.dsf.gdb 4 5012 2021-08-03 15:46:15.938
!MESSAGE Error in final launch sequence:

Failed to execute MI command:
-target-select remote localhost:2345/tcp
Error message from debugger back end:
localhost:2345/tcp: No such file or directory.
!STACK 1
org.eclipse.core.runtime.CoreException: Failed to execute MI command:
-target-select remote localhost:2345/tcp
Error message from debugger back end:
localhost:2345/tcp: No such file or directory.
	at org.eclipse.cdt.dsf.concurrent.Query.get(Query.java:112)
	at org.eclipse.cdt.dsf.gdb.launching.GdbLaunchDelegate.launchDebugSession(GdbLaunchDelegate.java:252)
	at org.eclipse.cdt.dsf.gdb.launching.GdbLaunchDelegate.launchDebugger(GdbLaunchDelegate.java:109)
	at org.eclipse.cdt.dsf.gdb.launching.GdbLaunchDelegate.launch(GdbLaunchDelegate.java:97)
	at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:803)
	at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:716)
	at org.eclipse.debug.internal.ui.DebugUIPlugin.buildAndLaunch(DebugUIPlugin.java:1021)
	at org.eclipse.debug.internal.ui.DebugUIPlugin$2.run(DebugUIPlugin.java:1224)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: java.lang.Exception: localhost:2345/tcp: No such file or directory.
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl$RxThread.processMIOutput(AbstractMIControl.java:1014)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl$RxThread.run(AbstractMIControl.java:843)
!SUBENTRY 1 org.eclipse.cdt.dsf.gdb 4 10004 2021-08-03 15:46:15.939
!MESSAGE Failed to execute MI command:
-target-select remote localhost:2345/tcp
Error message from debugger back end:
localhost:2345/tcp: No such file or directory.
!STACK 0
java.lang.Exception: localhost:2345/tcp: No such file or directory.
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl$RxThread.processMIOutput(AbstractMIControl.java:1014)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl$RxThread.run(AbstractMIControl.java:843)

PS: It looks like the real error is "localhost:2345/tcp: cannot resolve name: Servname not supported for ai_socktype:

MI output line: (gdb) 
MI output line: &"localhost:2345/tcp: cannot resolve name: Servname not supported for ai_socktype\n"
MI output line: 19^error,msg="localhost:2345/tcp: No such file or directory."
MI output line: (gdb) 
MI output line: 20^exit

A host `/usr/bin` folder mounted to `/var/run/host/usr/bin` is added to the `$PATH` in order to enable access to the programs installed on the host and not existing on the sandbox, for example, such as `gdbserver`. PATH is set to include '/var/run/host/usr/bin/' 'gdbinit' and 'gdbinit.d' are mounted to '/etc' to be accessed by gdb Host's '/etc/hosts' and '/etc/services' are mounted to '/etc' to be accessed by gdb '--device=all" permission is added to allow access to devices like /dev/pts/* from sandbox Issue: flathub#36 Signed-off-by: Victor Rubezhny <vrubezhny@redhat.com>

vrubezhny · 2021-08-09T20:19:44Z

The following bug has been created and the Gerrit change has been submitted for the localhost:2345/tcp: cannot resolve name: Servname not supported for ai_socktype error:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=575317

vrubezhny · 2021-08-10T12:00:18Z

The following fixes made to CDT restore the CDT debugger functionality:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=574874
https://bugs.eclipse.org/bugs/show_bug.cgi?id=575317

I believe we can close the issue when the fixed mentioned above are available in CDT release

'/tcp' suffix is not supported anymore with 'gdb', so an IP address should be in form of HOST:PORT (and not of HOST:PORT/PROTOCOL) See: flathub/org.eclipse.Java#36 Change-Id: I9eedc450da0d39305fc5d4cde851eb991c3c31f0 Signed-off-by: Victor Rubezhny <vrubezhny@redhat.com>

vrubezhny mentioned this issue Jul 7, 2021

[CDT] gdbserver not found #36 #45

Closed

Mailaender closed this as completed Dec 12, 2023

Mailaender added the bug Something isn't working label Apr 23, 2024

6b6279 mentioned this issue Sep 9, 2024

Debugging of JUnit test doesn't work due to network error #97

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CDT] gdbserver not found #36

[CDT] gdbserver not found #36

mcoufal commented Jun 2, 2021

vrubezhny commented Jul 8, 2021

vrubezhny commented Jul 8, 2021

vrubezhny commented Jul 9, 2021 •

edited

Loading

vrubezhny commented Jul 9, 2021 •

edited

Loading

vrubezhny commented Jul 9, 2021

jjohnstn commented Jul 10, 2021 via email

vrubezhny commented Jul 10, 2021 •

edited

Loading

jjohnstn commented Jul 10, 2021 via email

jjohnstn commented Jul 14, 2021

jjohnstn commented Jul 15, 2021

vrubezhny commented Aug 3, 2021 •

edited

Loading

vrubezhny commented Aug 9, 2021

vrubezhny commented Aug 10, 2021

[CDT] gdbserver not found #36

[CDT] gdbserver not found #36

Comments

mcoufal commented Jun 2, 2021

vrubezhny commented Jul 8, 2021

vrubezhny commented Jul 8, 2021

vrubezhny commented Jul 9, 2021 • edited Loading

vrubezhny commented Jul 9, 2021 • edited Loading

vrubezhny commented Jul 9, 2021

jjohnstn commented Jul 10, 2021 via email

vrubezhny commented Jul 10, 2021 • edited Loading

jjohnstn commented Jul 10, 2021 via email

jjohnstn commented Jul 14, 2021

jjohnstn commented Jul 15, 2021

vrubezhny commented Aug 3, 2021 • edited Loading

vrubezhny commented Aug 9, 2021

vrubezhny commented Aug 10, 2021

vrubezhny commented Jul 9, 2021 •

edited

Loading

vrubezhny commented Jul 9, 2021 •

edited

Loading

vrubezhny commented Jul 10, 2021 •

edited

Loading

vrubezhny commented Aug 3, 2021 •

edited

Loading