ENOTTY when using embedded Linux Console Window

[mqudsi]

Attempting to use VSLinux to debug applications that attempt to introspect stdin causes ENOTTY to be emitted by the remote linux kernel.

It should be reproducible with just this code sample:

#include <stdio.h>
#include <unistd.h>

int main(int argc, const char *argv[]) {
    int result = tcgetpgrp(STDIN_FILENO);
    printf("result: %d, errno: %d\n", result, errno);
}

The tcgetpgrp(3) call fails, errno is set to ENOTTY.

Is it possible to make debugging an application via the Linux Console Window transparent to such terminal-related kernel calls?

[lukka]

@mqudsi thanks for the report. We need to investigate on why the remote process does not have an allocated tty. Currently, when debugging, VS redirects the output of the debuggee using gdb redirection to another shell create for that purpose only, which has a pty allocated. It might be we have to reconsider this choice.

[mqudsi]

Thanks, @lukka.

Personally, I wouldn't rely on gdb to get the pty. It should be fairly straightforward to create a launcher that waits for gdb to connect then execs the target, then VS can start an SSH session, run that launcher, and attach gdb (so all debugging would use gdb attach instead of having gdb launch the target). That way you're guaranteed a native pty for the app and don't have to trust gdb to do the right thing.

[mqudsi]

It seems to me that the functionality I described in my last comment is now possible with the newer features available in launch.json, but I haven't dug deep enough into it to figure it out. I see that it's possible to override the launch command and to separately specify that gdb should attach to a process, I think?

[benmcmorran]

This issue is being closed because we are no longer tracking VS Linux issues on GitHub. If the issue still reproduces, bugs and new suggestions should be reported on the Developer Community forum. It is centralized, better equipped for group voting, and has more visibility.