Explore more efficient ways to scan for running web servers

[code-asher]

Currently we use node-netstat. Increasing the poll timeout to 5s greatly improved CPU load but it would be ideal if we could find a more efficient way to get the ports that have something listening on them.

[nhooyr]

Why not just connect to localhost HTTP ports every 5s? That should be really cheap.

[sr229]

if we're just checking for liveliness we can do this

ROM katacoda/docker-http-server:health
HEALTHCHECK --timeout=1s --interval=1s --retries=3 \
  CMD curl -s --fail http://localhost:80/ || exit 1

[sr229]

and regarding ports, I think we can just lock on a port using http nonetheless.

[code-asher]

The idea is to provide a list of listening web servers, so for example if you were to run webpack-dev-server on, say, port 9322, it would show up on the list and you could click it which would open a new tab so you could view the content that was being served on that port.

It would probably open up the /tunnel/<port> endpoint which just proxies to that port.

Maybe we don't need to scan at all, but in that case we might need some good way to educate users about the /tunnel endpoint.

Edit: actually we'd need something similar to the /tunnel endpoint except it can be opened in a browser window since I'm not sure /tunnel works that way, but the core scanning issue remains.

[nhooyr]

Why do we need to provide a list? User's explicitly run webservers anyway so they know exactly what port its running on. We should just use the common ports and then let them enter custom ports.

[coadler]

I agree with Anmol. There's also going to be a lot of noise in the list.

Here's a netstat of my work pc at idle:

tcp        0      0 0.0.0.0:34449           0.0.0.0:*      5820/spotify
tcp        0      0 0.0.0.0:57621           0.0.0.0:*      5820/spotify
tcp        0      0 localhost:6463          0.0.0.0:*      4743/Discord
tcp        0      0 localhost:mysql         0.0.0.0:*      7010/./cloud_sql_pr
udp        0      0 0.0.0.0:57621           0.0.0.0:*      5820/spotify
udp        0      0 0.0.0.0:bootpc          0.0.0.0:*      877/dhcpcd
udp        0      0 0.0.0.0:50861           0.0.0.0:*      5820/spotify
udp        0      0 0.0.0.0:ssdp            0.0.0.0:*      5820/spotify
udp        0      0 224.0.0.251:mdns        0.0.0.0:*      1292/chromium

Users probably wouldn't want to connect to these

[code-asher]

@coadler True, although none of that is running on ide.coder.com or in the Docker container if users are running it themselves. I could see it being an issue if someone is running code-server together with other services on their own server though, so I agree it's something we should consider.

@nhooyr What do you mean by letting them enter custom ports? Edit: I think the problem that needs solving is that users know the port, but they don't know the URL to get to that port (for example cdr.co as is currently used at ide.coder.com).

[code-asher]

One other use case to keep in mind while discussing possible solutions: For the local app, we depend on port scanning to know which ports to forward from the remote host to the local host. This lets you go to localhost:<port> just as you would with a local editor for a seamless experience.

[nhooyr]

@nhooyr What do you mean by letting them enter custom ports?

So by default you'd look at ports like 8080, 80 and whatnot (usual http ports0 for servers on the sidebar but also let the user enter their own custom ports in case they want those scanned too.

But I don't think port forwarding is the move. How about we just use a VPN and forward all localhost traffic onto their instance? Would be very efficient and make everything seamless.

[code-asher]

Ahh, interesting, I like that, and VPN sounds like a solid move to me.