Accept connections over HTTPS.

[sholladay]

This feature request is geared towards being able to run an HTTPS proxy server, as opposed to being able to connect through to HTTPS target pages on the other side, as is the goal of #4.

The Problem

• Presently, all client connections to hoxy are only possible over plain HTTP.
• In reverse mode, target pages that assume or demand HTTPS in their implementation may break completely because an HTTPS reverse proxy endpoint is not available.
• In forward mode, target pages that assume or demand HTTPS in their implementation will generally load their resources in a way that avoids proxying completely, because typically the OS routes HTTPS requests through an entirely different proxy configuration (via the $HTTPS_PROXY environment variable, for example, instead of $HTTP_PROXY) and in many cases this is out of the control of the user of hoxy's API. Setting both configurations to the same proxy will work in some scenarios when no https support #4 lands, but will break when the environment assumes (reasonably) that the proxy itself speaks HTTPS.

The Solution

• Provide an API to create a hoxy instance that will respond to HTTPS connections. This could be exposed through configuration on the existing options object given to Proxy or via a new class, as is common with other libraries.
• Ideally, all combinations of client and target protocols work by the time this is complete.

Client Protocol	Target Protocol	Proxy Direction	Works Today
HTTP	HTTP	Forward	✔️
HTTP	HTTPS	Forward	❌ (soon)
HTTPS	HTTP	Forward	❌
HTTPS	HTTPS	Forward	❌
HTTP	HTTP	Reverse	✔️
HTTP	HTTPS	Reverse	✔️
HTTPS	HTTP	Reverse	❌
HTTPS	HTTPS	Reverse	❌

[snoj]

Yay! What glorious tables!

[greim]

I'm going to try to argue that rows #3 and #4 can go away, and that #1 and #2 are good enough to cover all interesting forward proxying use cases.

[forward proxying] will break when the environment assumes (reasonably) that the proxy itself speaks HTTPS.

Maybe if I understood this part I'd come around, but it's the part I'm specifically unclear about. Which environment? And in what sense would it assume https? Neither OS X or Firefox proxy config screens for example have a "this proxy is itself an https server" checkbox. Thanks for bearing with me.

[sholladay]

So the issue with ignoring row 4 is Strict Transport Security, where the site demands a secure protocol. Most of the sites I work on use this and it's only going to get more popular. The end result is that the browser will refuse to display anything if the client protocol is plain HTTP.

Row 3 probably isn't so important. I don't have a strong need to upgrade the security of a target site through my proxy, although that would be cool. I included it mainly for completeness.

[greim]

HSTS guards against attempts to make anything other than "https://" appear in the URL bar. "https://" being in the URL bar depends not on whether the proxy runs as HTTPS, but on the conversation happening in the tunnel, which for our purposes is always going to to be HTTPS. So even though Hoxy always runs as HTTP, "https://" still appears in the URL bar and thus it shouldn't run afoul of HSTS.

[sholladay]

So even though Hoxy always runs as HTTP, "https://" still appears in the URL bar ...

I'll have to see this to believe it, I've never heard of a proxy setup where that is the case.

If you are correct, then fair enough, I will concede that we don't need row 4.

[snoj]

@sholladay, Could you be thinking of reverse proxies?

[greim]

Well, 2.0 is released now, so you can try it out directly against your HSTS servers. It would be good in any case to start getting more eyeballs on it, surfacing bugs, etc.

I think the reason an http server is allowed to mediate the tunnel is that it doesn't participate in the encryption, so there's really no need.

[sholladay]

Well how about them apples. I learned something new today. Thanks, guys, you are totally correct.

I am a bit confused about how this works underneath. Presumably the network layer needs to know the protocol of the proxy itself before connecting to it and my (incorrect) understanding was always that the whole point of having separate configurations for HTTP and HTTPS is to allow one to be assumed when an app (like a web browser) decides to route through that configuration (based on the protocol in the address bar, for example).

Now I'm left wondering what the consequences are of doing it like this (row 2) vs row 4.

[greim]

So to summarize the scope of this issue, given rows 1, 2, 5, 6 all now work and 3 and 4 can go away (with the caveat that Hoxy needs better documentation about how it intercepts http connect tunneling) the focus here is to get 7 and 8 working, agreed?

[sholladay]

Agreed. So @snoj, do you expect both row 7 and 8 to work with your version? Or is 8 a whole other level of project? And what is left to make a PR based on that?

[snoj]

e651fcb should do the trick for 8.....maybe 7 (hadn't tested that one).

[greim]

I think the only difference between 7 and 8 would be the value of the reverse option, which would simply trigger Hoxy's existing protocol switch like so:

// row 7
new hoxy.Proxy({
  reverse: 'http://example.com',
  key: ...,
  cert: ...
})

// row 8
new hoxy.Proxy({
  reverse: 'https://example.com',
  key: ...,
  cert: ...
})

[snoj]

@greim I agree, that should be the only difference. Again, my earlier commit was only tested under the 8 but I don't see why it wouldn't work for 7.

[greim]

Thanks @snoj. In my own tests it works for both 7 and 8. Published as 2.1.0. Let me know how it works.

PS: I shortened the option to just tls:

new hoxy.Proxy({
  reverse: 'https://www.google.com',
  tls: {
    key: fs.readFileSync(__dirname + '/my-server.key.pem'),
    cert: fs.readFileSync(__dirname + '/my-server.crt.pem'),
  }
});

[sholladay]

Thanks for all of the hard work on this, everyone. @greim if you create a new temporary ticket or re-open this, I would be happy to post another bounty against it. It really saves the day for me.