destination parameter -d behaves differently with local proxy vs remote proxy

[FrostKiwi]

This issue stems from the investigating done in #81. I'm very happy with my pure nginx setup which works in many challenging situations, but I believe there to be a bug in proxytunnel and it's -d parameter or a weird interaction with the intermediate proxy jump, which took a while to figure out.

I have ssh on Port 22 and had both used apache and now nginx with https://github.com/chobits/ngx_http_proxy_connect_module. Both nginx and apache were configured to encrypt with HTTPs and pass the HTTP_CONNECT packets 127.0.0.1:22 (which is not port forwarded).

When there is no corporate packet sniffing firewall in the way, the connection command looks like proxytunnel -E -p domain.com:443 -d 127.0.0.1:22. The header says:

Tunneling to 127.0.0.1:22 (destination)
Communication with local proxy:
 -> CONNECT 127.0.0.1:22 HTTP/1.1
 -> HOST: domain.com

Works for nginx and apache. Now we get to the corporate firewall case, which we tunnel through, so the command becomes proxytunnel -X -p corporate.com:8888 -r domain.com:443 -d 127.0.0.1:22. Here is where the weirdness begins. The header looks like:

Connected to corporate.com:8888 (local proxy)

Tunneling to domain.com:443 (remote proxy)
Communication with local proxy:
 -> CONNECT domain.com:443 HTTP/1.1
 -> Host: domain.com:443
 -> Proxy-Connection: Keep-Alive
 <- HTTP/1.1 200 Connection established

 <-

Set SNI hostname to domain.com

Tunneling to 127.0.0.1:22 (destination)
Communication with remote proxy:
 -> CONNECT 127.0.0.1:22 HTTP/1.1
 -> Host: 127.0.0.1:22
 -> Proxy-Connection: Keep-Alive
 <- HTTP/1.1 405 Not Allowed

HTTP return code: 405 Not Allowed
 <- Server: nginx

We land at the destination, but get denied. But looking at the communication with the remote proxy, we see domain.com:443 missing. For whatever reason, this is ok with apache, but not with nginx, both configured with the same ruleset.

Changing the destination parameter fixes this though. proxytunnel -X -p corporate.com:8888 -r domain.com:443 -d domain.com:22

Connected to corporate.com:8888 (local proxy)

Tunneling to domain.com:443 (remote proxy)
Communication with local proxy:
 -> CONNECT domain.com:443 HTTP/1.1
 -> Host: domain.com:443
 -> Proxy-Connection: Keep-Alive
 <- HTTP/1.1 200 Connection established

 <-

Set SNI hostname to domain.com

Tunneling to domain.com:22 (destination)
Communication with remote proxy:
 -> CONNECT domain.com:22 HTTP/1.1
 -> Host: domain.com:22
 -> Proxy-Connection: Keep-Alive
 <- HTTP/1.1 200 Connection Established

 <- Proxy-agent: nginx

Tunnel established.
SSH-2.0-OpenSSH_9.7

Now the HOST in the Header is correct and the connection works with nginx and the corporate firewall. Still, this is werid, as the destination is actually incorrect. There is no domain.com:22 to connect to, such a port is not open, only the local 127.0.0.1:22, but it works out. This must be some kind of bug in how -d works with local proxy vs remote proxy.

[e9hack]

I can't see a difference related to 'Now the HOST in the Header is correct'. I'm using a similar configuration:

ProxyCommand proxytunnel -v -X -m -P xxxxx -p my-company-proxy.com:8080 -R xxxxxx -r proxy.my-domain.net:443 -d %h:%p -C ~/.ssh/certs -H "User-Agent: 
Mozilla/4.0 (compatible; MSIE 6.0; Win32)"

The result is:

$ ssh z-box
SSL local to remote proxy enabled
Use credential manager for passwords
Connected to my-company-proxy.com:8080 (local proxy)

Tunneling to proxy.my-domain.net:443 (remote proxy)
Communication with local proxy:
-> CONNECT proxy.my-domain.net:443 HTTP/1.1
-> Host: proxy.my-domain.net:443
-> Proxy-Connection: Keep-Alive
-> User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Win32)
<- HTTP/1.1 200 Connection established

<-

Set SNI hostname to proxy.my-domain.net

Tunneling to z-box.my-local-net.lan:22 (destination)
Communication with remote proxy:
-> CONNECT z-box.my-local-net.lan:22 HTTP/1.1
-> Host: z-box.my-local-net.lan:22
-> Proxy-Authorization: Basic xxxxxxxxxxxxxxxxxx
-> Proxy-Connection: Keep-Alive
-> User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Win32)
<- HTTP/1.0 200 Connection Established

<- Proxy-agent: Apache

<-

1

Tunnel established.

Linux z-box 6.10.11-custom-1 #5 SMP PREEMPT_DYNAMIC Sat Oct  5 18:56:03 CEST 2024 x86_64

Please ignore option -m and output 'Use credential manager for passwords'. This my extension to proxytunnel.

[sge-d-o]

@FrostKiwi,

I did some research on this with the following outcome.

Firstly, according to https://www.rfc-editor.org/rfc/rfc9110#CONNECT and https://www.rfc-editor.org/rfc/rfc9112#name-authority-form

CONNECT 127.0.0.1:22 HTTP/1.1
Host: domain.com

is incorrect while

CONNECT 127.0.0.1:22 HTTP/1.1
Host: 127.0.0.1:22

is correct.

Secondly, Apache at first evaluates the Host header field to decide which virtual host configuration to apply. Nginx seem to follow the same approach according to https://nginx.org/en/docs/http/request_processing.html.

Thirdly, Apache implicitly knows of a default server or virtual host. From https://httpd.apache.org/docs/2.4/vhosts/examples.html:

[...]That means that if a request is received that does not match one of the specified ServerName directives, it will be served by this first .

while Nginx seems to know of an implicit as well as of an explicit default server (see https://nginx.org/en/docs/http/request_processing.html).

With that said

proxytunnel -E -p domain.com:443 -d 127.0.0.1:22 with

Tunneling to 127.0.0.1:22 (destination)
Communication with local proxy:
 -> CONNECT 127.0.0.1:22 HTTP/1.1
 -> HOST: domain.com

works for Apache and Nginx because both first evaluate the Host header field and then process the CONNECT request according to the domain.com vhost configuration. It is not RFC-compliant, though.

proxytunnel -X -p corporate.com:8888 -r domain.com:443 -d 127.0.0.1:22 with

Connected to corporate.com:8888 (local proxy)

Tunneling to domain.com:443 (remote proxy)
Communication with local proxy:
 -> CONNECT domain.com:443 HTTP/1.1
 -> Host: domain.com:443
 -> Proxy-Connection: Keep-Alive
 <- HTTP/1.1 200 Connection established

 <-

Set SNI hostname to domain.com

Tunneling to 127.0.0.1:22 (destination)
Communication with remote proxy:
 -> CONNECT 127.0.0.1:22 HTTP/1.1
 -> Host: 127.0.0.1:22
 -> Proxy-Connection: Keep-Alive

works for Apache if domain.com is the first (or even only) vhost. It picks the domain.com vhost's configuration although 127.0.0.1:22 is requested with the Host header field.
I can only assume your configuration for domain.com is not the first name-based virtual server (again, see https://nginx.org/en/docs/http/request_processing.html), thus Nginx fails.

Please explicitly declare your virtual server domain.com as default server (again, see https://nginx.org/en/docs/http/request_processing.html) and test again.

[FrostKiwi]

Please explicitly declare your virtual server domain.com as default server (again, see https://nginx.org/en/docs/http/request_processing.html) and test again.

RIGHT ON THE MONEY! 🎉 That was it! Now specifying -d 127.0.0.1:22 instead of -d domain.com:22 works, as it does with Apache.
So the NGINX case really is a bit more minute in configuration.

proxytunnel -X -p corporate.com:8888 -r domain.com:443 -d 127.0.0.1:22

Connected to corporate.com:8888 (local proxy)

Tunneling to domain.com:443 (remote proxy)
Communication with local proxy:
 -> CONNECT domain.com:443 HTTP/1.1
 -> Host: domain.com:443
 -> Proxy-Connection: Keep-Alive
 <- HTTP/1.1 200 Connection established

 <-

Set SNI hostname to domain.com

Tunneling to 127.0.0.1:22 (destination)
Communication with remote proxy:
 -> CONNECT 127.0.0.1:22 HTTP/1.1
 -> Host: 127.0.0.1:22
 -> Proxy-Connection: Keep-Alive
 <- HTTP/1.1 200 Connection Established

 <- Proxy-agent: nginx

Tunnel established.
SSH-2.0-OpenSSH_9.7

In my case it's NixOS and the config looks like this, where default inserts the required default_server

services = {
  ...
  nginx = {
    ...
    virtualHosts = {
     "domain.com" =  {
       enableACME = true;
       forceSSL = true;
       default = true;

       extraConfig = ''
         proxy_connect;
         proxy_connect_allow 22;
         proxy_connect_address 127.0.0.1;
       '';
      };
      ...
    };
    ...
  };
  ...
};

Also from the docs you linked:

In the configuration above, the default server is the first one — which is nginx’s standard default behaviour.

This explains, why I had it working when I tested it on another server in isolation, where only one server block existed.

[sge-d-o]

Happy I could help. 🙂

[e9hack]

The issue with destination as host instead of the proxy is triggered in http.c line 171:

		sprintf( buf, "CONNECT %s HTTP/1.1\r\nHost: %s\r\n", args_info.dest_arg, args_info.host_arg ? args_info.host_arg : args_info.dest_arg);

args_info.dest_arg must be replace with args_info.remproxy_arg.

[sge-d-o]

@e9hack,

args_info.dest_arg must be replace with args_info.remproxy_arg.

It must not. The current behaviour complies with the RFCs. Please re-read the Firstly section from my message above.