Understanding worker initialization and connection

[wlandau]

From #208 (comment) and https://mschubert.github.io/clustermq/articles/technicaldocs.html#worker-startup, I understand this part of the worker initialization process:

1. The scheduler launches a new job.
2. The job launches R and calls clustermq:::worker("IP_OF_LOCAL_PROCESS") to connect back to the local R process.

After that, how exactly does the tunneling and reverse tunneling work? I am not sure I understand mrc-ide/rrq#64 (comment):

In clustermq the tunnel is established local>remote, so no need to expose any local ports (but the remote needs to accept SSH connections and reverse tunneling; traffic over the tunnel is always encrypted via SSH)

For the tunneling that happens from the local machine to the remote machine, is that a step after (2)? I thought (2) involved a new SSH connection from the remote process to the local process.

[mschubert]

There are two options, either the workers are on the same network (then they need to know the host ip/port to connect to), or they are on a separate network (then they need to know the ip/port of the remote end of the tunnel).

The tunneling is "step 0", so it occurs before launching workers. It is active before because it launches the proxy, which in turn launches the workers. The tunnel and proxy will stay active because they forward worker traffic to/from the host (using two reverse tunnels, one for controlling the proxy and one for forwarding traffic).

This is why, in the SSH.tmpl file, you can see:

clustermq/inst/SSH.tmpl

Lines 2 to 3 in 7812577

	-R {{ ctl_port }}:localhost:{{ local_port }} \
	-R {{ job_port }}:localhost:{{ fwd_port }} \

Workers behind a proxy (tunnel) need to connect to the fwd_port of the tunnel on the remote end. So there is no need to expose any ports from the user to the remote.

[wlandau]

Still wrapping my head around which ports are on which computers and which computers initiate the tunnels.

[mschubert]

I think you're misunderstanding the difference between a forward (-L) and a reverse tunnel (-R). In both cases, the SSH connection is established local>remote. With a forward tunnel, I can locally connect to a port that will be forwarded to the remote. With a reverse tunnel, a remote process can connect to a remote port that will be forwarded to the local machine.

Does that help?

[wlandau]

It does help, thanks. So the tunnel that forwards traffic needs to be a reverse tunnel because the workers need to find a port on the proxy that will forward data from the proxy to the local machine (fwd_port?).

[wlandau]

Then in what sense are connections bidirectional?

[mschubert]

Once the connection is established, both peers can send/receive data (with some limitations imposed by ZMTP)