Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remote development over SSH introduced bug when upgrading from VS Code 1.51.1 to 1.52.0 #4205

Closed
chloehale opened this issue Dec 16, 2020 · 17 comments
Assignees
Labels
*duplicate Issue identified as a duplicate of another issue(s)

Comments

@chloehale
Copy link

I'm not sure how to narrow down the issue, but the Remote Development extension pack works great with VS Code version 1.51.1 and only sometimes works with VS Code version 1.52.0.

Here's my setup:

  • VS Code version 1.52.0 running on macOS Catalina (10.15.7).
  • Code repository hosted on a Linux machine running Ubuntu 18.04.5 LTS.
  • I'm connecting between the two machines using SSH.

Here's what I'm seeing:

  • When I have no services running on my Linux machine, the Remote Development over SSH works great. Everything is snappy and responsive.
  • When I have services running on my Linux machine, VS Code hangs really badly. I can still SSH into the machine just fine and the Remote Development extension appears to be able to make the connection alright, but I can't open or search for files, run other types of searches, or save edits. Everything seems to be stuck in a loading state. I do get the following logs in the output under Log (Window) over and over again:
[2020-12-16 15:19:51.912] [renderer1] [info] [remote-connection][ExtensionHost][0f570…][reconnect] resolving connection...
[2020-12-16 15:19:51.912] [renderer1] [info] [remote-connection][ExtensionHost][0f570…][reconnect] connecting to 127.0.0.1:64734...
[2020-12-16 15:19:51.943] [renderer1] [info] [remote-connection][ExtensionHost][0f570…][reconnect] reconnected!
[2020-12-16 15:20:11.951] [renderer1] [info] [remote-connection][ExtensionHost][0f570…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
[2020-12-16 15:20:11.951] [renderer1] [info] [remote-connection][ExtensionHost][0f570…][reconnect] waiting for 5 seconds before reconnecting... 

What I've tried so far (to no avail):

  • Disabled all other VS Code extensions except "Remote Development" and "Remote - SSH" (just to make sure the other extensions weren't the culprit).
  • Set "remote.autoForwardPorts": false since I noticed that starting services on my Linux machine caused many more ports to be forwarded.
  • Set "git.enabled": false and disabled everything I could find related to git to rule git problems out.

No matter what I've tried, I can consistently reproduce this issue: no services running = no lag, services running = VS Code hangs. The only thing that I've found to fix it thus far is to go back to using VS Code version 1.51.1.

Is there any information I can provide to help figure out what the root cause is?

@github-actions github-actions bot added the ssh Issue in vscode-remote SSH label Dec 16, 2020
@roblourens
Copy link
Member

Which services are you running? Is there any process using lots of CPU or memory on the remote?

@chloehale
Copy link
Author

I have about 12 Scala services running for my company's development environment. These services use a lot of CPU on the Linux machine when spinning up (takes about 1 minute). Once they're spun up, they're using ~20-30% of CPU and a very minimal amount of memory, but VS Code still hangs on the host Mac (no matter how much time I give it). The Linux machine itself is not hanging and I can still perform other actions without lag (even when SSH'ing into it from my host).

@roblourens
Copy link
Member

Can you try with the 1.52.1 we released yesterday, or Insiders? https://code.visualstudio.com/insiders

@alexdima any idea why we would be more sensitive to this in 1.52?

@roblourens roblourens added the info-needed Issue requires more information from poster label Dec 17, 2020
@chloehale
Copy link
Author

Okay, I just tried with both 1.52.1 and Insiders (1.53.0-insider). No luck. Both of those behave exactly the same as described in the original post for 1.52.0.

As a bit of clarification to my original post where I'd included 5 lines of logs that keep repeating...that repeating only happens when the services are running on my remote machine. I don't get repeating logs like that if my services aren't running. So maybe there's an issue deep down with the connection not staying open?

Thanks for your help thus far.

@roblourens
Copy link
Member

Could you try setting this "remote.autoForwardPorts": false to disable a new feature that we added this month? Would like to see if that makes a difference.

@chloehale
Copy link
Author

That was actually one of the first settings I changed while troubleshooting. See second bullet point under What I've tried so far section in my original post. Turning it to false didn't help.

@alexdima
Copy link
Member

@chloehale Could you please share with us the server logs? I think you can find them on the remote machine at $HOME/.vscode-remote.

@chloehale
Copy link
Author

I do not see any files under the name .vscode-remote Here are some logs that I am seeing...let me know if I can dig up any others (and where to find them).

Log (Remote Server) in VS Code's output panel

[2020-12-18 11:55:46.738] [remoteagent] [info] 

*
* Visual Studio Code Server
*
* Reminder: You may only use this software with Visual Studio family products,
* as described in the license https://aka.ms/vscode-remote/license
*


[2020-12-18 11:55:46.738] [remoteagent] [info] Extension host agent started.
[2020-12-18 11:55:47.036] [remoteagent] [info] [127.0.0.1][7676fec7][ManagementConnection] New connection established.
[2020-12-18 11:55:47.039] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] New connection established.
[2020-12-18 11:55:47.082] [remoteagent] [info] Listing 0 persistent terminals, 0 total terminals
[2020-12-18 11:55:47.177] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] <23783> Launched Extension Host Process.
[2020-12-18 11:56:15.413] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 11:56:43.880] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 11:57:11.411] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 11:57:37.934] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 11:58:04.258] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 11:58:29.753] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 11:58:54.856] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 11:59:23.314] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 11:59:49.450] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:00:15.752] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:00:42.137] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:01:08.673] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:01:34.775] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:02:01.206] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:02:27.459] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:02:53.654] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:03:19.876] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:03:46.277] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:04:12.438] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:04:38.889] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:05:05.014] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:05:31.224] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:05:57.443] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:06:22.535] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:06:49.969] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:07:16.202] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:07:42.505] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:08:08.821] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:08:34.929] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:09:01.063] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:09:26.146] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:09:51.193] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:10:16.490] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:10:20.867] [remoteagent] [info] [127.0.0.1][7676fec7][ManagementConnection] The client has disconnected gracefully, so the connection will be disposed.
[2020-12-18 12:10:23.170] [remoteagent] [info] [127.0.0.1][c6d9b717][ManagementConnection] New connection established.
[2020-12-18 12:10:23.170] [remoteagent] [info] [127.0.0.1][80b27ebf][ExtensionHostConnection] New connection established.
[2020-12-18 12:10:23.174] [remoteagent] [info] [127.0.0.1][80b27ebf][ExtensionHostConnection] <27552> Launched Extension Host Process.
[2020-12-18 12:10:23.190] [remoteagent] [info] Listing 0 persistent terminals, 0 total terminals
[2020-12-18 12:10:51.771] [remoteagent] [info] [127.0.0.1][80b27ebf][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:11:17.346] [remoteagent] [info] [127.0.0.1][80b27ebf][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:11:48.458] [remoteagent] [info] [127.0.0.1][c6d9b717][ManagementConnection] The client has disconnected gracefully, so the connection will be disposed.
[2020-12-18 12:11:52.531] [remoteagent] [info] [127.0.0.1][d6df8131][ManagementConnection] New connection established.
[2020-12-18 12:11:52.531] [remoteagent] [info] [127.0.0.1][d8b75f90][ExtensionHostConnection] New connection established.
[2020-12-18 12:11:52.534] [remoteagent] [info] [127.0.0.1][d8b75f90][ExtensionHostConnection] <28028> Launched Extension Host Process.
[2020-12-18 12:11:52.547] [remoteagent] [info] Listing 0 persistent terminals, 0 total terminals
[2020-12-18 12:12:20.467] [remoteagent] [info] [127.0.0.1][d8b75f90][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:12:24.386] [remoteagent] [info] [127.0.0.1][80b27ebf][ExtensionHostConnection] <27552> Extension Host Process exited with code: 0, signal: null.
[2020-12-18 12:12:46.117] [remoteagent] [info] [127.0.0.1][d8b75f90][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:13:11.165] [remoteagent] [info] [127.0.0.1][d8b75f90][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:13:55.086] [remoteagent] [info] [127.0.0.1][d8b75f90][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:14:25.313] [remoteagent] [info] [127.0.0.1][d8b75f90][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:14:53.216] [remoteagent] [info] [127.0.0.1][d8b75f90][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:15:18.286] [remoteagent] [info] [127.0.0.1][d8b75f90][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:15:19.328] [remoteagent] [info] [127.0.0.1][d6df8131][ManagementConnection] The client has disconnected gracefully, so the connection will be disposed.
[2020-12-18 12:16:12.707] [remoteagent] [info] [127.0.0.1][fdad81c1][ManagementConnection] New connection established.
[2020-12-18 12:16:12.707] [remoteagent] [info] [127.0.0.1][00fde852][ExtensionHostConnection] New connection established.
[2020-12-18 12:16:12.710] [remoteagent] [info] [127.0.0.1][00fde852][ExtensionHostConnection] <31453> Launched Extension Host Process.
[2020-12-18 12:16:12.732] [remoteagent] [info] Listing 0 persistent terminals, 0 total terminals
[2020-12-18 12:16:21.358] [remoteagent] [info] [127.0.0.1][45f974c8][ExtensionHostConnection] <23783> Extension Host Process exited with code: 0, signal: null.
[2020-12-18 12:16:40.475] [remoteagent] [info] [127.0.0.1][00fde852][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:17:06.015] [remoteagent] [info] [127.0.0.1][00fde852][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:17:38.018] [remoteagent] [info] [127.0.0.1][00fde852][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:18:04.591] [remoteagent] [info] [127.0.0.1][00fde852][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:18:30.190] [remoteagent] [info] [127.0.0.1][00fde852][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:18:57.639] [remoteagent] [info] [127.0.0.1][00fde852][ExtensionHostConnection] The client has reconnected.
[2020-12-18 12:19:24.156] [remoteagent] [info] [127.0.0.1][00fde852][ExtensionHostConnection] The client has reconnected.

Remote - SSH log in VS Code's output panel

[12:16:03.724] Log Level: 1
[12:16:03.725] remote-ssh@0.62.0
[12:16:03.725] darwin x64
[12:16:03.726] SSH Resolver called for "ssh-remote+192.168.86.211", attempt 1
[12:16:03.726] "remote.SSH.useLocalServer": true
[12:16:03.727] "remote.SSH.sshPath": undefined
[12:16:03.727] "remote.SSH.sshConfigurationFile": undefined
[12:16:03.727] "remote.SSH.useFlock": true
[12:16:03.727] "remote.SSH.lockfilesInTmp": false
[12:16:03.727] "remote.SSH.localServerDownload": auto
[12:16:03.727] "remote.SSH.remoteServerListenOnSocket": false
[12:16:03.727] "remote.SSH.showLoginTerminal": false
[12:16:03.727] "remote.SSH.maxReconnectionAttempts": 9007199254740991
[12:16:03.728] SSH Resolver called for host: 192.168.86.211
[12:16:03.728] Setting up SSH remote "192.168.86.211"
[12:16:03.730] Acquiring local install lock: /var/folders/qy/pln7m_0j3w77xy9ydzq6fkjc0000gp/T/vscode-remote-ssh-192.168.86.211-install.lock
[12:16:03.732] Looking for existing server data file at /Users/chloe/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-192.168.86.211-ea3859d4ba2f3e577a159bc91e3074c5d85c0523-0.62.0/data.json
[12:16:03.733] Using commit id "ea3859d4ba2f3e577a159bc91e3074c5d85c0523" and quality "stable" for server
[12:16:03.736] Install and start server if needed
[12:16:03.741] Checking ssh with "ssh -V"
[12:16:03.753] > OpenSSH_8.1p1, LibreSSL 2.7.3

[12:16:03.758] askpass server listening on /var/folders/qy/pln7m_0j3w77xy9ydzq6fkjc0000gp/T/vscode-ssh-askpass-daad36b61d943ae2f540ca07becea280e40e988e.sock
[12:16:03.758] Spawning local server with {"ipcHandlePath":"/var/folders/qy/pln7m_0j3w77xy9ydzq6fkjc0000gp/T/vscode-ssh-askpass-0fd6b3ad54b296cec905c58622287270d54773df.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","64803","-o","ConnectTimeout=15","192.168.86.211"],"dataFilePath":"/Users/chloe/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-192.168.86.211-ea3859d4ba2f3e577a159bc91e3074c5d85c0523-0.62.0/data.json"}
[12:16:03.758] Local server env: {"DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/chloe/.vscode/extensions/ms-vscode-remote.remote-ssh-0.62.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app/Contents/MacOS/Code Helper (Renderer)","VSCODE_SSH_ASKPASS_MAIN":"/Users/chloe/.vscode/extensions/ms-vscode-remote.remote-ssh-0.62.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/qy/pln7m_0j3w77xy9ydzq6fkjc0000gp/T/vscode-ssh-askpass-daad36b61d943ae2f540ca07becea280e40e988e.sock"}
[12:16:03.762] Spawned 47484
[12:16:03.856] > local-server> Spawned ssh: 47488
[12:16:03.860] stderr> OpenSSH_8.1p1, LibreSSL 2.7.3
[12:16:03.861] stderr> debug1: Reading configuration data /Users/chloe/.ssh/config
[12:16:03.861] stderr> debug1: /Users/chloe/.ssh/config line 21: Applying options for 192.168.86.211
[12:16:03.861] stderr> debug1: Reading configuration data /etc/ssh/ssh_config
[12:16:03.861] stderr> debug1: /etc/ssh/ssh_config line 47: Applying options for *
[12:16:03.861] stderr> debug1: Connecting to 192.168.86.211 [192.168.86.211] port 22.
[12:16:03.875] stderr> debug1: fd 3 clearing O_NONBLOCK
[12:16:03.875] stderr> debug1: Connection established.
[12:16:03.875] stderr> debug1: identity file /Users/chloe/.ssh/chloe.pem type 0
[12:16:03.875] stderr> debug1: identity file /Users/chloe/.ssh/chloe.pem-cert type -1
[12:16:03.876] stderr> debug1: Local version string SSH-2.0-OpenSSH_8.1
[12:16:03.876] stderr> debug1: Remote protocol version 2.0, remote software version OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
[12:16:03.876] stderr> debug1: match: OpenSSH_7.6p1 Ubuntu-4ubuntu0.3 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
[12:16:03.876] stderr> debug1: Authenticating to 192.168.86.211:22 as 'chloe'
[12:16:03.876] stderr> debug1: SSH2_MSG_KEXINIT sent
[12:16:03.880] stderr> debug1: SSH2_MSG_KEXINIT received
[12:16:03.880] stderr> debug1: kex: algorithm: curve25519-sha256
[12:16:03.881] stderr> debug1: kex: host key algorithm: ecdsa-sha2-nistp256
[12:16:03.881] stderr> debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
[12:16:03.881] stderr> debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
[12:16:03.882] stderr> debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
[12:16:03.894] stderr> debug1: Server host key: ecdsa-sha2-nistp256 SHA256:qDx0EyuGdH0lbryXpbYM3NE/tHzKRFFiw5rGdTzYmJE
[12:16:03.895] stderr> debug1: Host '192.168.86.211' is known and matches the ECDSA host key.
[12:16:03.895] stderr> debug1: Found key in /Users/chloe/.ssh/known_hosts:3
[12:16:03.897] stderr> debug1: rekey out after 134217728 blocks
[12:16:03.897] stderr> debug1: SSH2_MSG_NEWKEYS sent
[12:16:03.897] stderr> debug1: expecting SSH2_MSG_NEWKEYS
[12:16:03.897] stderr> debug1: SSH2_MSG_NEWKEYS received
[12:16:03.897] stderr> debug1: rekey in after 134217728 blocks
[12:16:03.897] stderr> debug1: Will attempt key: /Users/chloe/.ssh/chloe.pem RSA SHA256:UJBOTi94hperLAH/noPA5KJZtWabK6DYj8ymCoGr+C0 explicit agent
[12:16:03.897] stderr> debug1: SSH2_MSG_EXT_INFO received
[12:16:03.897] stderr> debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
[12:16:03.908] stderr> debug1: SSH2_MSG_SERVICE_ACCEPT received
[12:16:03.914] stderr> debug1: Authentications that can continue: publickey
[12:16:03.914] stderr> debug1: Next authentication method: publickey
[12:16:03.914] stderr> debug1: Offering public key: /Users/chloe/.ssh/chloe.pem RSA SHA256:UJBOTi94hperLAH/noPA5KJZtWabK6DYj8ymCoGr+C0 explicit agent
[12:16:03.921] stderr> debug1: Server accepts key: /Users/chloe/.ssh/chloe.pem RSA SHA256:UJBOTi94hperLAH/noPA5KJZtWabK6DYj8ymCoGr+C0 explicit agent
[12:16:03.944] stderr> debug1: Authentication succeeded (publickey).
[12:16:03.944] stderr> Authenticated to 192.168.86.211 ([192.168.86.211]:22).
[12:16:03.944] stderr> debug1: Local connections to LOCALHOST:64803 forwarded to remote address socks:0
[12:16:03.945] stderr> debug1: Local forwarding listening on ::1 port 64803.
[12:16:03.945] stderr> debug1: channel 0: new [port listener]
[12:16:03.945] stderr> debug1: Local forwarding listening on 127.0.0.1 port 64803.
[12:16:03.945] stderr> debug1: channel 1: new [port listener]
[12:16:03.945] stderr> debug1: channel 2: new [client-session]
[12:16:03.945] stderr> debug1: Requesting no-more-sessions@openssh.com
[12:16:03.945] stderr> debug1: Entering interactive session.
[12:16:03.945] stderr> debug1: pledge: network
[12:16:04.189] stderr> debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
[12:16:04.195] stderr> debug1: Sending environment.
[12:16:04.201] > Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-42-generic x86_64)
> 
>  * Documentation:  https://help.ubuntu.com
>  * Management:     https://landscape.canonical.com
>  * Support:        https://ubuntu.com/advantage
> 
>  * Introducing self-healing high availability clusters in MicroK8s.
>    Simple, hardened, Kubernetes for production, from RaspberryPi to DC.
> 
>      https://microk8s.io/high-availability
> 
>  * Canonical Livepatch is available for installation.
>    - Reduce system reboots and improve kernel security. Activate at:
>      https://ubuntu.com/livepatch
> 
> 306 packages can be updated.
> 201 updates are security updates.
> 
> Your Hardware Enablement Stack (HWE) is supported until April 2023.
[12:16:04.213] > ready: 0702b817db88
[12:16:04.220] > Linux 5.4.0-42-generic #46~18.04.1-Ubuntu SMP Fri Jul 10 07:21:24 UTC 2020
[12:16:04.220] Platform: linux
[12:16:04.236] > 0702b817db88: running
[12:16:04.250] > Acquiring lock on /home/chloe/.vscode-server/bin/ea3859d4ba2f3e577a159bc91e3074c5d85c0523/vscode-remote-lock.chloe.ea3859d4ba2f3e577a159bc91e3074c5d85c0523
[12:16:04.252] > Installing to /home/chloe/.vscode-server/bin/ea3859d4ba2f3e577a159bc91e3074c5d85c0523...
[12:16:04.254] > 0702b817db88%%1%%
> Downloading with wget
[12:16:11.943] > Download complete
[12:16:11.947] > 0702b817db88%%2%%
> tar --version:
> tar (GNU tar) 1.29
> Copyright (C) 2015 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> 
> Written by John Gilmore and Jay Fenlason.
[12:16:12.511] > Checking /home/chloe/.vscode-server/.ea3859d4ba2f3e577a159bc91e3074c5d85c0523.log and /home/chloe/.vscode-server/.ea3859d4ba2f3e577a159bc91e3074c5d85c0523.pid for a running server
> Looking for server with pid: 23726
[12:16:12.546] > Found running server...
>  
> *
> * Reminder: You may only use this software with Visual Studio family products,
> * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057)
> *
>  
[12:16:12.546] > Checking server status on port 46443 with wget
> 0702b817db88: start
> sshAuthSock====
> webUiAccessToken====
> listeningOn==46443==
[12:16:12.548] > osReleaseId==ubuntu==
> arch==x86_64==
> tmpDir==/run/user/1000==
> platform==linux==
> unpackResult==success==
> didLocalDownload==0==
> downloadTime==7690==
> installTime==560==
> extInstallTime====
> serverStartTime====
> 0702b817db88: end
[12:16:12.548] Received install output: 
sshAuthSock====
webUiAccessToken====
listeningOn==46443==osReleaseId==ubuntu==
arch==x86_64==
tmpDir==/run/user/1000==
platform==linux==
unpackResult==success==
didLocalDownload==0==
downloadTime==7690==
installTime==560==
extInstallTime====
serverStartTime====

[12:16:12.549] Remote server is listening on 46443
[12:16:12.549] Parsed server configuration: {"remoteListeningOn":{"port":46443},"osReleaseId":"ubuntu","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"","tmpDir":"/run/user/1000","platform":"linux","downloadTime":7690,"installTime":560}
[12:16:12.550] Persisting server connection details to /Users/chloe/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-192.168.86.211-ea3859d4ba2f3e577a159bc91e3074c5d85c0523-0.62.0/data.json
[12:16:12.551] Starting forwarding server. localPort 64847 -> socksPort 64803 -> remotePort 46443
[12:16:12.551] Forwarding server listening on 64847
[12:16:12.551] Waiting for ssh tunnel to be ready
[12:16:12.554] [Forwarding server 64847] Got connection 0
[12:16:12.555] Tunneled 46443 to local port 64847
[12:16:12.555] Resolved "ssh-remote+192.168.86.211" to "127.0.0.1:64847"
[12:16:12.568] TELEMETRY: {"eventName":"resolver","properties":{"outcome":"success","osReleaseId":"ubuntu","arch":"x86_64","askedPw":"0","askedPassphrase":"0","asked2fa":"0","askedHostKey":"0","remoteInConfigFile":"0","gotUnrecognizedPrompt":"0","dynamicForwarding":"1","localServer":"1"},"measures":{"resolveAttempts":1,"retries":1,"timing.totalResolveTime":8841,"timing.preSshTime":32,"timing.establishSshTime":478,"timing.scriptTime":8791,"timing.downloadTime":7690,"timing.installTime":560}}
[12:16:12.568] ------


[12:16:12.642] [Forwarding server 64847] Got connection 1
[12:16:12.642] [Forwarding server 64847] Got connection 2
[12:16:13.245] [Forwarding server 64847] Got connection 3
[12:16:14.026] Unhandled rejection: Error: spawn git ENOENT
	at Process.ChildProcess._handle.onexit (internal/child_process.js:264:19)
	at onErrorNT (internal/child_process.js:456:16)
	at processTicksAndRejections (internal/process/task_queues.js:81:21)
[12:16:14.072] Unhandled rejection: Error: spawn git ENOENT
	at Process.ChildProcess._handle.onexit (internal/child_process.js:264:19)
	at onErrorNT (internal/child_process.js:456:16)
	at processTicksAndRejections (internal/process/task_queues.js:81:21)
[12:16:35.533] TELEMETRY: {"eventName":"extraInfo","properties":{"remoteCloud":""},"measures":{}}
[12:16:40.424] [Forwarding server 64847] Got connection 4
[12:17:05.904] [Forwarding server 64847] Got connection 5
[12:17:37.971] [Forwarding server 64847] Got connection 6
[12:18:04.449] [Forwarding server 64847] Got connection 7
[12:18:30.086] [Forwarding server 64847] Got connection 8
[12:18:57.585] [Forwarding server 64847] Got connection 9

@alexdima
Copy link
Member

This might be caused by our new automatic port forwarding feature which scans the remote OS for available ports in order to forward them locally (microsoft/vscode#112843)

@alexr00
Copy link
Member

alexr00 commented Dec 21, 2020

@chloehale would you be willing to try the following steps? This would help us confirm that we've found the issue, and you'll also get to use a more up-to-date version.

  1. On your remote machine, add the following line to your .bashrc file: export VSCODE_DISABLE_PROC_READING=true
  2. Install VS Code Insiders
  3. Try to repro your original issue with VS Code Insiders.

Setting that environment variable will cause VS Code (insiders only right now, I'll make a proper fix that doesn't involve an environment variable for the next stable release), to not do the proc filesystem reading that we do to automatically forward ports for you. This does mean that automatic port forwarding will not work while you have that environment variable set.

@chloehale
Copy link
Author

I just followed your steps and am unable to reproduce the hang/lag issue. It appears to have worked!

Maybe you've already included the fix in the Insiders version that I downloaded ~20 minutes ago, but I thought I'd mention that (oddly) I'm now unable to reproduce my issue with the Insiders version at all...even if I set export VSCODE_DISABLE_PROC_READING=false. I had been consistently reproducing it on the Insiders version a few days ago.

@alexr00
Copy link
Member

alexr00 commented Dec 23, 2020

Thanks for trying it out! We haven't added a more permanent fix yet, but maybe the issue is just transient. A more permanent fix will come in January, when we start releasing insiders every day again.

@filipesilva
Copy link

Hi there,

I'm seeing something similar in BetterThanTomorrow/calva#869 (comment). For convenience, I'll paste the relevant comments here:


BetterThanTomorrow/calva#869 (comment)

I can confirm that going back to VS Code 1.51 (October version) as instructed in https://stackoverflow.com/questions/49346733/how-to-downgrade-vscode makes editing fast again.


BetterThanTomorrow/calva#869 (comment)

Heya, I think I spoke too soon. I can see the slowness happen again, but I don't think this is inherently related to Calva - it's just that calvafmt might be a bit demanding on the CPU when editing.

Let me explain a bit better. My setup is Calva while running a Shadow CLJS build, and I am not using a repl connection to Calva.

When using VSCode 1.51.1, task manager shows WSL as using 0-2% CPU. But when using VSCode 15.2.1, there's a constant 12-19% CPU usage. If I turn off the Shadow CLJS server or close VSCode, this drops to 0-2%.

#4205 details similar slowness with the Remote WSL extention on the same version change.

If I follow the steps in #4205 (comment), I get a 0-2% CPU usage at rest. But if I remove that var, I see a 15%-20% CPU usage.

So I think what's happening here is that the VSCode remote extension is not playing well with Shadow CLJS.

@astrowonk
Copy link

astrowonk commented Jan 11, 2021

Python unit tests done over Remote SSH also show a bunch of ports in the range 30000-40000 that start showing up in the Ports view as of late. I wrote this up in #4274. And auto port forwarding from processes I deliberately launch in the terminal that I want to forward, don't forward. I imagine the underlying cause could be similar. I'm not having any performance issues otherwise though. Just the ports issue.

@alexdima alexdima assigned alexr00 and unassigned alexdima Jan 12, 2021
@mshindal
Copy link

Just adding another data point - I had this same problem. Everything worked fine in 1.51.1, but the problem began in 1.52.0 and remained in 1.52.1. When I ran a web service that used ports in the 8000's range, VSCode stopped working.

The fix mentioned above also worked for me -

  1. Kill the existing vscode-server processes on the remote machine. The official instructions here did not work for me, I had to use pkill -f vscode
  2. Add export VSCODE_DISABLE_PROC_READING=true to my .bashrc (well it's .zshrc for me but still)
  3. Use VSCode Insiders
  4. Use Remote - SSH (Nightly) extension

@alexr00
Copy link
Member

alexr00 commented Jan 21, 2021

Thanks for the additional data point! If you're on the latest insiders, then VSCODE_DISABLE_PROC_READING isn't needed anymore. The long term fix is in, so it should "just work".

@alexr00 alexr00 added *duplicate Issue identified as a duplicate of another issue(s) and removed info-needed Issue requires more information from poster ssh Issue in vscode-remote SSH labels Jan 21, 2021
@alexr00 alexr00 closed this as completed Jan 21, 2021
@ghost
Copy link

ghost commented Feb 14, 2021

I'm commenting on this issue because I've suddenly started seeing this issue! It happens when code starts and tries to open the remote workspace.

Version: 1.54.0-insider (user setup)
Commit: 6ac9a3ecb3698e82bf901f11bbb5940f6bc3c197
Date: 2021-02-12T05:13:16.111Z
Electron: 11.2.3
Chrome: 87.0.4280.141
Node.js: 12.18.3
V8: 8.7.220.31-electron.0
OS: Windows_NT x64 10.0.19041

and the errors I'm seeing:

[19:09:57.944] Log Level: 2
[19:09:57.947] remote-ssh@0.64.0
[19:09:57.947] win32 x64
[19:09:57.947] SSH Resolver called for "ssh-remote+code-server.myserver.com", attempt 1
[19:09:57.947] "remote.SSH.useLocalServer": false
[19:09:57.948] "remote.SSH.showLoginTerminal": false
[19:09:57.948] "remote.SSH.remotePlatform": {"code-server.myserver.com":"linux"}
[19:09:57.948] "remote.SSH.sshPath": undefined
[19:09:57.948] "remote.SSH.sshConfigurationFile": undefined
[19:09:57.948] "remote.SSH.useFlock": true
[19:09:57.948] "remote.SSH.lockfilesInTmp": false
[19:09:57.948] "remote.SSH.localServerDownload": auto
[19:09:57.948] "remote.SSH.remoteServerListenOnSocket": false
[19:09:57.948] "remote.SSH.showLoginTerminal": false
[19:09:57.948] "remote.SSH.defaultExtensions": []
[19:09:57.949] SSH Resolver called for host: code-server.myserver.com
[19:09:57.949] Setting up SSH remote "code-server.myserver.com"
[19:09:57.957] Using commit id "6ac9a3ecb3698e82bf901f11bbb5940f6bc3c197" and quality "insider" for server
[19:09:57.959] Install and start server if needed
[19:09:57.961] Checking ssh with "ssh -V"
[19:09:57.985] > OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5

[19:09:57.987] Running script with connection command: ssh -T -D 49889 "code-server.myserver.com" bash
[19:09:57.988] Terminal shell path: C:\Windows\System32\cmd.exe
[19:09:59.465] > d46eccacc21a: running
> �]0;C:\Windows\System32\cmd.exe�
[19:09:59.465] Got some output, clearing connection timeout
[19:09:59.523] > Acquiring lock on /home/code-server/.vscode-server-insiders/bin/6ac9a3ecb3698e82
> bf901f11bbb5940f6bc3c197/vscode-remote-lock.code-server.6ac9a3ecb3698e82bf901f11
> bbb5940f6bc3c197
[19:09:59.552] > Found existing installation at /home/code-server/.vscode-server-insiders/bin/6ac
> 9a3ecb3698e82bf901f11bbb5940f6bc3c197...
> Checking /home/code-server/.vscode-server-insiders/.6ac9a3ecb3698e82bf901f11bbb5
> 940f6bc3c197.log and /home/code-server/.vscode-server-insiders/.6ac9a3ecb3698e82
> bf901f11bbb5940f6bc3c197.pid for a running server
> Looking for server with pid: 14964
[19:09:59.655] > Found running server...
> 
> *
> * Reminder: You may only use this software with Visual Studio family products, 
> * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057)
> *
> 
> Checking server status on port 43293 with wget
> d46eccacc21a: start
> sshAuthSock====
> webUiAccessToken====
> listeningOn==43293==
> osReleaseId==ubuntu==
> arch==x86_64==
> tmpDir==/run/user/1002==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime====
> connectionToken==1111111a-a1aa-11aa-1111-aa1111111111==
> d46eccacc21a: end
[19:09:59.656] Received install output: 
sshAuthSock====
webUiAccessToken====
listeningOn==43293==
osReleaseId==ubuntu==
arch==x86_64==
tmpDir==/run/user/1002==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime====
connectionToken==1111111a-a1aa-11aa-1111-aa1111111111==
[19:09:59.656] Remote server is listening on 43293
[19:09:59.656] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":43293},"osReleaseId":"ubuntu","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"","tmpDir":"/run/user/1002","platform":"linux","connectionToken":"1111111a-a1aa-11aa-1111-aa1111111111"},"installUnpackCode":""}
[19:09:59.657] Starting forwarding server. localPort 49895 -> socksPort 49889 -> remotePort 43293
[19:09:59.657] Forwarding server listening on 49895
[19:09:59.657] Waiting for ssh tunnel to be ready
[19:09:59.658] Tunneled 43293 to local port 49895
[19:09:59.658] Resolved "ssh-remote+code-server.myserver.com" to "127.0.0.1:49895"
[19:09:59.658] [Forwarding server 49895] Got connection 0
[19:09:59.665] ------
[19:09:59.672] [Forwarding server 49895] Got connection 1
[19:09:59.682] > 
[19:09:59.898] [Forwarding server 49895] Got connection 2
[19:10:01.385] [Forwarding server 49895] Got connection 3
[19:10:02.558] Unhandled rejection: Error: spawn \home\code-server\code\git ENOENT
	at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19)
	at onErrorNT (internal/child_process.js:469:16)
	at processTicksAndRejections (internal/process/task_queues.js:84:21)
[19:10:02.561] Unhandled rejection: Error: spawn \home\code-server\code\git ENOENT
	at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19)
	at onErrorNT (internal/child_process.js:469:16)
	at processTicksAndRejections (internal/process/task_queues.js:84:21)
[19:10:04.561] Unhandled rejection: Error: spawn C:\Users\andrew\AppData\Local\Programs\Microsoft VS Code Insiders\docker ENOENT
	at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19)
	at onErrorNT (internal/child_process.js:469:16)
	at processTicksAndRejections (internal/process/task_queues.js:84:21)
[19:10:04.563] Unhandled rejection: Error: spawn C:\Users\andrew\AppData\Local\Programs\Microsoft VS Code Insiders\docker ENOENT
	at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19)
	at onErrorNT (internal/child_process.js:469:16)
	at processTicksAndRejections (internal/process/task_queues.js:84:21)
[19:10:04.563] Unhandled rejection: Error: spawn C:\Users\andrew\AppData\Local\Programs\Microsoft VS Code Insiders\docker ENOENT
	at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19)
	at onErrorNT (internal/child_process.js:469:16)
	at processTicksAndRejections (internal/process/task_queues.js:84:21)
[...continues forever...]

I've tried exporting VSCODE_DISABLE_PROC_READING=true in my .bashrc with no luck. Does anyone have any suggestions? Thanks !

@github-actions github-actions bot locked and limited conversation to collaborators Mar 7, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
*duplicate Issue identified as a duplicate of another issue(s)
Projects
None yet
Development

No branches or pull requests

7 participants