Stuck when connecting remote server using Remote Development while remote server is reachable in ssh (random nodes assigned)

[zhliang-work]

• VSCode Version: 1.34.20-insider
• OS Version: Local Client - Ubuntu 16.04, Remote Server - Ubuntu 16.04

I am using a group server and when I try to connect it using the new Remote Development extension, the SSH tunnel process stalls. But I can use normal SSH to connect to the server.

The terminal outputs are as follows:

> bash: cannot set terminal process group (21617): Inappropriate ioctl for device
> bash: no job control in this shell
> Found existing installation...
> Found running server...
>  
> *
> * Reminder: You may only use this software with Visual Studio family products,
> * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057)
> *
>  
> 8ab17bf7-eebb-479a-a4d8-c0813ff9ec5c==46161==
"install" terminal command done
Received install output: 8ab17bf7-eebb-479a-a4d8-c0813ff9ec5c==46161==
Server is listening on port 46161
Spawning tunnel with: ssh  a18 -L localhost:42233:localhost:46161 "echo -e 'Connected to SSH Host - Please do not close this terminal' && sleep infinity"
Spawned SSH tunnel between local port 42233 and remote port 46161
Waiting for ssh tunnel to be ready
Tunneling remote port 46161 to local port 42233
Resolving "ssh-remote+a18" to "localhost:42233", attempt: 1
> channel 3: open failed: connect failed: Connection refused
> channel 4: open failed: connect failed: Connection refused
> channel 5: open failed: connect failed: Connection refused
> Connected to SSH Host - Please do not close this terminal

Does this issue occur when all extensions are disabled?: Yes

[roblourens]

the SSH tunnel process stalls

Can you explain what the problem is exactly? The log doesn't necessarily show an issue (Connection refused may or may not be an issue)

[zhliang-work]

The problem is that I cannot create a new bash session or open any file in the remote group server in my university. In detail, when I tried to create a new bash session or open any file on the group server, VS Code has no response.
So I suspect that the SSH connection cannot be established correctly (I am not sure because the terminal tells me VS Code has connected to SSH Host).

[Update]

When I tried it on a Google Cloud server, it works. So there might be some problems on the university server. However, I cannot see any error information from the Remote Development extension. I don't know what to do.

[safelix]

Hi there, I experienced the same behavior as @SaltedFishLZ when trying to access an HPC cluster.

As the log suggests, the connection seems to be established. However, there is no response when I try to open Folder/Files. When I start a new terminal, it will open a new terminal without any contents.

I tried to investigate a little more on this by examining and debugging parts of %TEMP%\vscode-linux-multi-line-command-myHost.sh . After a while, I figured that the $PORT numbers did not always match:

> VSCode Output: Remote - SSH
...
Received install output: ababba7d-6b8f-447b-816b-0ddc7762d999==$PORT==
...

> cat "$HOME/.vscode-remote/.$COMMIT_ID.log" | grep -E 'Extension host agent listening on [0-9]+' | grep -v grep | grep -o -E '[0-9]+'
$PORT

> lsof -i
COMMAND   PID    USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
node    19279 user 13u  IPv6 38349751      0t0  TCP localhost:$PORT->localhost:$client_PORT (ESTABLISHED)

I then realized that the cluster, obviously, does some kind of load-balancing. By using ssh user@cluster.domain.com there is no guarantee that I repeatedly connect to a specific node.

During the connection setup, the extension uses multiple ssh connections to the host, assuming that the domain is always resolved to the same machine. I am able to successfully start the server-script and successfully establish an ssh-tunnel but since the assumption does not hold, the local client doesn't receive any response.

I was able to circumvent the load-balancing, by accessing a specific node with ssh user@node.cluster.domain.com. However, it would be nice if the server initialization and tunnel-establishing could be done in one step. But this is a question of what kind of systems you want to support. Also, a little bit more documentation and possibilities to modify the existing script would have saved me quite some time ;)

Thanks for this great extension, it makes my life much easier!

TL;DR
The extension assumes that a domain is always resolved to the same machine, which is not the case with clusters. Solution might be to access nodes directly.

[roblourens]

That's really interesting! I think your analysis is correct and would bet that @SaltedFishLZ is seeing the same thing.

I would like to get down to opening just one ssh connection but it's tricky. Another workaround for you might be using the "ControlMaster" feature, https://code.visualstudio.com/docs/remote/troubleshooting#_enabling-alternate-ssh-authentication-methods

[safelix]

Yes, I would expect the "ControlMaster" feature to be a much cleaner workaround. Unfortunately, I'm using OpenSSH on Windows where this features is not supported (yet). PowerShell/Win32-OpenSSH#1328

Yes, doing it in one ssh connection without Control Master seems to be very tricky. This is what I found on the topic, but you probably already consider this: https://unix.stackexchange.com/questions/30515/how-to-setup-port-redirection-after-a-ssh-connection-has-been-opened