Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

no https support #4

Closed
greim opened this issue Dec 5, 2010 · 60 comments
Closed

no https support #4

greim opened this issue Dec 5, 2010 · 60 comments

Comments

@greim
Copy link
Owner

greim commented Dec 5, 2010

Hoxy currently doesn't support https proxying. In order to be useful, it would need to be able to see content passing through UNencrypted. I'm not sure if there's a proxying scheme supported by browsers that makes this possible, and even if there was, I'm not sure I'd want to do it. On the off chance there's a proxying scheme supported by browsers that allows just HTTP headers through as clear text, but not the content bodies, that might be an acceptable compromise. Need to do more research.

@greim greim mentioned this issue Aug 21, 2014
@sholladay
Copy link
Contributor

@greim I have posted a rather large bounty for this issue.
https://www.bountysource.com/issues/7974725-no-https-support

@greim
Copy link
Owner Author

greim commented Jan 23, 2015

Wow. That's really cool and I think the fix would be quite beneficial. I'll try to carve out some time soon to tackle this.

I just want to clarify that even if I or someone manages to get this working, there will be some unavoidable painfulness with regard to certificates. Namely, hoxy users will need to create a self-signed certificate and swap it in place of the actual cert. Browsers will then (correctly) generate dire warnings since this is effectively a MITM attack on an encrypted channel.

@sholladay
Copy link
Contributor

Yeah, my benchmark for this will be how Charles does it: http://www.charlesproxy.com/documentation/using-charles/ssl-certificates/

But if there's anything we can do to make it easier, I will sponsor that as well.

@Phoenixmatrix
Copy link

Just stumbled on this while looking at what was available out there (I'm doing similar, except the code is pathetically bad, so i'm always on the lookout for a library that does it cleaner, hehe).

You can use Pem or spawn with OpenSSL to generate certificates on the fly... first you need to create a CA (and it needs to conform to certain standards, else Chrome won't take it...and it HAS to be a self signed CA signing your per-domain certs, else headers like strict transport won't allow it (ie: it won't work for gmail, for example). Then its just a matter of generating the certificates on the fly.

you'll need to use sniCallback to be able to stick with only 1 https server that works for multiple hosts (other libraries that support SSL create 1 https server per host name...ugh...), which makes things a little tricky. Node 10.X only supports synchronous sniCallback... 0.11.3 supports asynchronous, which you need, but its unstable. 0.11.4 and up (including io.js) will work fine.

Then its just a matter of having your http proxy listen to CONNECT verb and pipe them over to your https one, and you're good to go. You need some kind of caching though, because openssl will trash Windows machines when generating certs.

I'm not in the business of writing a proxy, I just did it because no one else did it correctly (for https support that allow sniffing/modifying... node-http-proxy supports https, but they don't expose the right hooks and it would be a pain to make it do so), and I really like your API...

So feel free to look at my (terrible, but working) implementation: https://github.com/Phoenixmatrix/phoenixmatrix-proxy/tree/master/js/lib (proxy.js and certificate.js) and use it as your starting point for the research. Then I'd be able to use your proxy for my projects instead :)

@greim
Copy link
Owner Author

greim commented Feb 5, 2015

Thanks for all this info, I'll definitely take a look!

@magicode118
Copy link

Was also looking for something like this for HTTPS requests so that I can monitor the GET requests a particular HTTPS link is logging. Charles does it and was hoping to integrate this to my Protractor/Node tests!

@sholladay
Copy link
Contributor

Automated testing is one of my use cases as well. I work on a lot of 3rd party JS libraries, the enterprise SaaS sort of thing. Figuring out if our next release will break in the customer environment is tricky and before hoxy we were manually maintaining our own repository of customer website clones, so that we could inject any code necessary for testing. Now we simply spin up a machine with a hoxy instance and it takes care of the rest. But customers that use HTTPS require manual testing and bad hacks with Charles.

@snoj
Copy link

snoj commented Jun 23, 2015

9381088 Rudimentary https support that probably needs some fleshing out. Passes existing tests but new https based tests will be needed. Doing an old fashioned hand test in my browser showed me that a slightly modified example worked just fine though. Well...so long as you ignore the warnings about a bad/invalid certificate.

(run "npm install pem" to use this example.)

var hoxy = require('./');
var pem = require('pem');
pem.createCertificate({commonName: "localhost", days:7, selfSigned:true}, function(err, keys) {
  var proxy = new hoxy.Proxy({
    reverse: "https://snoj.us"
    ,tls: {key: keys.serviceKey, cert: keys.certificate}
  });

  proxy.intercept({
    phase: 'response',
    mimeType: 'text/html',
    as: '$'
  }, function(req, resp){
    resp.$('title').text('Unicorns.');
  });

  proxy.listen(8080);
})

@nerdbeere
Copy link
Contributor

Awesome @snoj!
I'm currently building a free alternative to charles based on electron and https support really would be a cool feature.
Unfortunately your example just crashes electron without any error message.

@snoj
Copy link

snoj commented Jun 23, 2015

@nerdbeere You might need to turn on logging to find out what's failing.

proxy.log('error warn debug');

Just wild guessing here, might it be a certificate issue? @Phoenixmatrix mentioned some stuff about needing certain settings for a certificate to accomplish some stuff. I can't seem to get pem to output anything but a V1 certificate. Granted I'm fairly new to pem so it could just be user error.

[edit] I was using pem for quickly testing the code and to give an easy example. You should be able to use your own certificates easily.

Also, I didn't test hoxy as a generic proxy. It was only tested as a reverse proxy.

@snoj
Copy link

snoj commented Jun 24, 2015

c8a1403 Straight up proxy is working...sort of. Entire unencrypted stream will be available for hoxy magic once the hoxy magic is piped into the new magic.

Code cleanup, tests and some more flesh is still needed though...but it's getting there!

@greim
Copy link
Owner Author

greim commented Jun 24, 2015

@snoj were you actually able to get a cleartext stream on an http request using these changes? (i.e. the title said "unicorns" and the url bar said "https" simultaneously?) I don't see anything listening to connect in the code, which my current set of assumptions is that you'd have to pipe connects to the server (as explained by @Phoenixmatrix).

@Phoenixmatrix I'm currently trying to get a proof of concept working, completely independently of hoxy, so that when it's time to implement I have a blueprint of sorts. Below is the current state of my latest attempt at a POC, using your suggestions and @snoj's pem example. This proxies https pages normally and prints CONNECT for https pages, which of course then hang in the browser.

Need to flesh out the connect handler obviously, but more importantly I think there needs to be a tls server which is what would use the cert data and act as the MITM, then that would just pipe everything as cleartext into the http proxy server. Anyway that's my working theory at the moment; suggestions/corrections welcome.

import https from 'https'
import http from 'http'
import url from 'url'
import pem from 'pem'

pem.createCertificate({days:1,selfSigned:true}, (err, keys) => {
  let proxy = http.createServer((fromClient, toClient) => {
    let purl = url.parse(fromClient.url)
    let h = purl.protocol === 'http:' ? http : https
    let toServer = h.request({
      hostname: purl.hostname,
      port: purl.port,
      method: fromClient.method,
      path: purl.path,
      headers: fromClient.headers,
    }, fromServer => {
      toClient.writeHead(fromServer.statusCode, fromServer.headers)
      fromServer.pipe(toClient)
    })
    fromClient.pipe(toServer)
  })
  proxy.listen(8080)
  console.log('proxy listening on 8080')
  proxy.on('connect', (request, socket, head) => {
    console.log(request.method)
  })
})

(type babel foo.js | node to run this.)

@snoj
Copy link

snoj commented Jun 24, 2015

@greim My first code commit only worked with reverse proxying for https which did work with the intercept feature of hoxy. The browser showed the address "https://localhost:8080" and the title showed "Unicorns."

The second commit does work as a regular proxy. (Using main example in readme.md, unmodified.) However it doesn't hook into the hoxy stuff like intercept yet. Right now it creates really bad self-signed certificates on the fly that are supplied to the end client (browser, wget, or whatever). In the end, I'm planning on letting the user supply a CA so that just in time certificates can be trusted.

General idea of the flow:

  1. client (browser) sends HTTP CONNECT {targethost}:{targetport} to hoxy
  2. hoxy connects to the {targethost}:{targetport} to grab certificate details.
  3. create certificate with actual certificate details.
  4. create new tmp https server with certificate details on a random port.
  5. pipe client https request to tmp https server
  6. tmp https pipes the request to the actual target.

Mostly this is a proof of concept. Eventually I hope to convert things to use the SNICallback feature.

The CONNECT verb doesn't work when the proxy is https based. If this isn't a limitation of HTTPS itself, it might be either node or that I was using a certificate that wasn't trusted by the system. In other words (when using bash and wget on ubuntu), export https_proxy="https://localhost:8080" doesn't work, while export https_proxy="http://localhost:8080" does.

@greim
Copy link
Owner Author

greim commented Jun 24, 2015

The CONNECT verb doesn't work when the proxy is https based

That's consistent with what I was seeing too, which is why I thought the proxy could just remain an http server, which does receive CONNECT events when browsers use it as their "https proxy". Then have a tls server on the side which handles the cert stuff but otherwise pipes cleartext to the http server. I think otherwise we're on similar tracks.

@snoj
Copy link

snoj commented Jun 25, 2015

After hacking away last night at an SNICallback version, I learned that it might not work out. The problem is passing along the target port to the private https instance. I can't think of a way to do so without mucking around with the byte stream in unholy ways.

@snoj
Copy link

snoj commented Jun 26, 2015

Well after some driving around this last evening, I figured out a way to do the single https/SNICallback without doing unholy things to the byte stream.

The issue I'm facing now though is passing off the data though Cycle.

Relevant commit fe08bcf.

@snoj
Copy link

snoj commented Jun 26, 2015

5fb5ff1 intercept is now working on https traffic. However there is bug where not everything is passed. Things seem to be cut up and I'm not sure why at the moment.

example/test js

var hoxy = require('./');
var pem = require('pem');
pem.createCertificate({commonName: "localhost", days:7, selfSigned:true}, function(err, keys) {
  var proxy = new hoxy.Proxy({
    tlspassthru: {key: keys.serviceKey, cert: keys.certificate}
  });

  proxy.intercept({
    phase: 'response',
    mimeType: 'text/html',
    as: '$'
  }, function(req, resp){
    resp.$('title').text('Unicorns.')
  });

  proxy.log('error warn debug info', process.stdout);

  proxy.listen(8080);
})

@snoj
Copy link

snoj commented Jun 26, 2015

Hmm it's looking like it's the intercept that is the cause of the incomplete transfer. (It doesn't matter what is done in the intercept, it merely needs to be defined.)

I'll try looking into this more tomorrow. For now, it's off to bed for me!

@greim
Copy link
Owner Author

greim commented Jun 29, 2015

@snoj - How's your research coming? I did manage to get a proof of concept running that works in Firefox after some experimentation, but I couldn't make it work with the pem lib (there went a few hours of my life) so ended up exec-ing openssl commands directly. One thing you mention:

hoxy connects to the {targethost}:{targetport} to grab certificate details

Am I missing some critical aspect, or why is that necessary? It was working for me without doing that.

@snoj
Copy link

snoj commented Jun 29, 2015

@greim, I did get somewhere...well more info but no resolution. Anything that returned over 32768 caused cheerio to choke. The specific wget error was Read error at byte 32768/39852 (Success). Retrying. Any request less than that (for instance github.com) doesn't choke cheerio.

Connecting to the remote server was to get details on the far end's certificate to feed back to the client when I was making a new https server for each request. It might be able to be reworked.

@snoj
Copy link

snoj commented Jun 30, 2015

Update, issue appears to be not with cheerio, but somewheres else. The following code works, but also only results in a portion of the whole document. I think tomorrow I'll look into the streams.

var hoxy = require('./');
var pem = require('pem');
pem.createCertificate({commonName: "localhost", days:7, selfSigned:true}, function(err, keys) {
  var proxy = new hoxy.Proxy({
    tlspassthru: {key: keys.serviceKey, cert: keys.certificate}
  });

  proxy.intercept({
    phase: 'response',
    mimeType: 'text/html'
    //,as: '$'
  }, function(req, resp){
    resp.$ = require('cheerio').load(resp._source.read().toString());
    resp.$('title').text('Unicorns.');
  });

  proxy.log('error warn debug info', process.stdout);

  proxy.listen(8080);
})

@sholladay
Copy link
Contributor

@snoj I have been experiencing issues suspiciously similar to what you are describing about partial documents, when running as intercepts on Node 0.12.x instances. I haven't gotten around to debugging or making a simplified test case, so haven't filed any issues. But humor me here, I'm curious what node --version is for you, and if it is 0.12.x, whether using 0.10.x fixes the problem.

@snoj
Copy link

snoj commented Jun 30, 2015

My laptop is at node version 0.12.5. I'll try getting version 0.10.39 installed tonight and post the results.

@snoj
Copy link

snoj commented Jul 1, 2015

Well...that is interesting. I downloaded 10.39 on both my windows and ubuntu instances and the windows version worked wonderfully. The linux version...not so much. Apparently there is a difference between the the two OS versions. The linux version doesn't have support for tls.createSecureContext nor has support for async SNICallback.

@sholladay
Copy link
Contributor

Sounds like my guess has a decent chance of being correct. Namely, that your implementation is fine but there's some kind of upstream bug, possibly in Node core, that we need to workaround or patch.

@greim
Copy link
Owner Author

greim commented Jul 6, 2015

One of my excuses for not advancing this project more quickly is that its internals are heavily asynchronous, which despite using promises still takes me a long time to re-wrap my brain around.

I have a rewrite finished which keeps the API identical but makes the async stuff simpler by using promises + generators (and babel to keep it working in non-es6 engines). I'd like to release that as v2.0. Then, having somewhat de-hairball-ified the project, follow up shortly with v2.1 containing https support, which at that point should be easy to add based on work done by everyone in this thread.

I'll get that published in a day-ish and keep this thread updated.

@snoj
Copy link

snoj commented Jul 6, 2015

As an addendum to that, after hacking away at odd times this last US holiday week, it would seem that the issue lies somewhere in the _sendToClient() area.

But I'm going to wait on @greim to publish his new innards and then start again from that.

@snoj
Copy link

snoj commented Jul 8, 2015

Hrrrrmmmmm. Dies around let responseFromServer = yield partiallyFulfilledRequest.receive() in lib/proxy.js.

Well, that's all I'm going to do for tonight.

@snoj
Copy link

snoj commented Jul 8, 2015

While doing some business I remembered that I didn't add rejectUnauthorized: false to relevent sections of lib/request.js.

Assuming I get home at a decent time tonight, I'll do that and try again.

@snoj
Copy link

snoj commented Jul 8, 2015

Had a few minutes. Added, quickly tested, and found to work. 642422e

By the by, haven't tested the https reverse proxy in a while....so it might not be working.

@sholladay
Copy link
Contributor

So the expectation is that I generate and import a certificate (via pem or other means) and point hoxy to it like this, right?

var proxy = new hoxy.Proxy(
    {
        reverse : 'https://sitecues.com',
        tlspassthru : {
            key  : keys.serviceKey,
            cert : keys.certificate
        }
    }
);

I haven't been able to get any response over HTTPS with that, so the answer may be that reverse is broken, or I'm missing something.

@snoj
Copy link

snoj commented Jul 10, 2015

@sholladay, that would be correct for my most recent commit. I did not write or test https based reverse proxy functionality.

I will rectify that soon though.

@snoj
Copy link

snoj commented Jul 10, 2015

6452345 Fixes a very embarrassing bug.

@snoj
Copy link

snoj commented Jul 10, 2015

e651fcb reverse https proxy is working again.

var hoxy = require('./');
var pem = require('pem');
pem.createCertificate({commonName: "localhost", days:7, selfSigned:true}, function(err, keys) {
  var proxy = new hoxy.Proxy({
    reverse: "https://snoj.us"
    ,tlsopts: {key: keys.serviceKey, cert: keys.certificate}
  });

  proxy.intercept({
    phase: 'response',
    mimeType: 'text/html',
    as: '$'
  }, function(req, resp){
    resp.$('title').text('Unicorns.');
  });
  proxy.log('error warn debug info', process.stdout);

  proxy.listen(8080);
})

@snoj
Copy link

snoj commented Jul 10, 2015

Now to flesh out some certificate authority stuff. Might not happen tonight though. Depends on children and another task I need to get done by tomorrow.

@snoj
Copy link

snoj commented Jul 10, 2015

ac69a7d CA stuff is started, but horribly broken though. Everything else may or may not work. Haven't tested and I have to do some tasks now.

@greim
Copy link
Owner Author

greim commented Jul 10, 2015

I'd assumed the success criteria here was direct proxying, since reverse proxying to an https host has (presumably?) worked all along. My goal was to be able to generate a self-signed key/cert combo, tell Firefox (or whatever) to trust it as a CA, launch hoxy like so:

var proxy = new hoxy.Proxy({
  certAuthority: {
    cert: fs.readFileSync( ... ),
    key: fs.readFileSync( ... ),
  },
})

...then browse to sites directly. (In the background hoxy uses the CA to spoof certs for every host.) That's what I've got working in the POC and in the rewrite branch at the moment, but running hoxy itself as an https server isn't required for any of the above and seems like a different use case that could wait for later. But tell me if I'm wrong :)

@greim
Copy link
Owner Author

greim commented Jul 10, 2015

On OS X Chrome uses the system's trusted root CAs, so simply double clicking my self-signed crt.pem from the finder opened a prompt to add it as a root CA. Firefox has a separate cert manager IIRC, and I haven't yet tried it with any other clients.

@sholladay
Copy link
Contributor

We can separate out the concepts of proxying to and serving over HTTPS as two different issues if that helps anyone. I can confirm that http -> https reverse proxying has been working fine.

I'm doing the crazy thing of running hoxy to service a major front-end feature on a corporate website. 0:) So I need https -> https in reverse mode and that's my own top priority. But I want to help with all of the other use cases in any way possible, since we also rely upon it in forward mode during development for all kinds of things.

@greim
Copy link
Owner Author

greim commented Jul 10, 2015

We can separate out the concepts of proxying to and serving over HTTPS as two different issues if that helps anyone.

I think it clears things up for me to do that, since in my mind they're different use cases and otherwise mutually-exclusive. Otherwise the rewrite is basically done and direct https proxying works, but not https -> https reverse proxying. I'll try to get 2.0 published soon and maybe this thread can move over to a new thread focused on the latter use case.

@snoj
Copy link

snoj commented Jul 10, 2015

Works for me.

@sholladay
Copy link
Contributor

10-4, makes sense to me.

I went ahead and created #44 for client connections, so this can solely be focussed on the target.

@sholladay
Copy link
Contributor

I'm not totally sure if I captured the expectations of this ticket properly over there, so lay the smackdown if necessary, @greim :)

@snoj
Copy link

snoj commented Jul 20, 2015

@greim, @Phoenixmatrix, @sholladay, How do you all wanna split the bounty?

@greim
Copy link
Owner Author

greim commented Jul 20, 2015

Three way even split? @sholladay do the current https features meet your original need?

@greim greim closed this as completed Jul 20, 2015
@sholladay
Copy link
Contributor

I will be doing more extensive testing next week. But so far things seem good. If I had to nitpick, I'd ask for better documentation. It talks a bit about the how, but not much on what or why. That will make it unintuitive if I come back to it after a while of not using it.

An example thought might be, "the certAuthority and tls options are extremely similar, what is the difference between them and is there a situation where I need both?"

As for the bounty, no strong feelings, but an even split sounds good to me. @snoj has made a claim and I'll accept it (next week) with the understanding that he will take care of that.

@snoj
Copy link

snoj commented Jul 24, 2015

I'll rescind my claim until you're happy then.

As for the split, I figured there was a way for the backer to plot things if multiple people made a claim.

@greim
Copy link
Owner Author

greim commented Jul 24, 2015

Cool. Okay, so this is a rough draft of something that will eventually live in the docs. Let me know if it clears things up sufficiently so that I can tune it:

How does HTTPS proxying work in Hoxy?

First, let's review how HTTPS proxying works in general, outside the context of Hoxy. Suppose you want to insert a proxy between yourself and the HTTPS website "https://example.com". There are two ways to accomplish this: direct proxying and reverse proxying.

Direct HTTPS proxying

To set up a direct HTTPS proxy, launch the proxy on port 8080 and set your browser to use localhost:8080 as its HTTPS proxy. (This is typically a separate config option from normal HTTP proxying.) Then type "https://example.com" directly into your browser.

What happens here is different from what happens in HTTP proxying. During HTTP proxying, the proxy receives requests like this:

GET http://www.example.com/foo.html HTTP/1.1

In English this might read as "please go to example.com and get /foo.html for me." But during HTTPS proxying, the proxy receives requests like this:

CONNECT example.com:443 HTTP/1.1

Which in English might read as "please connect me to example.com, I want to have a private conversation." And so the proxy connects them. The client will then initiate a TLS handshake with the server on that connection. Then, it may send traffic like this over it:

GET /foo.html HTTP/1.1

...however the proxy can't see this, since it's encrypted. Anything might happen on that connection, for all the proxy can tell.

This is formally known as HTTP CONNECT tunneling. Since the proxy isn't privy to the conversation, there's no need for it to be running as an HTTPS server itself, even though "https://" appears in the browser's URL bar. Its job is just to shovel TCP packets (which happen to contain undecypherable TLS traffic) back and forth.

Reverse HTTPS proxying

A reverse HTTPS proxy is a different animal altogether. To set it up, launch the proxy on port 8080. Since this is a reverse HTTPS proxy, you'll also need to provide a few additional startup options:

  1. A reverse proxy target, consisting of https://example.com, AKA the server being proxied to.
  2. A private key.
  3. A certificate. This certificate MUST contain 'localhost' as its common name, as clients will require this to match "localhost:8080" in the URL bar.

The key and the cert in particular are necessary because, from the browser's point of view, a reverse proxy isn't a proxy at all, but a regular HTTPS webserver. And regular HTTPS webservers need a key/cert to identify themselves and do TLS. Thus, the proxy doesn't respond to CONNECT requests, and otherwise isn't relegated to shoveling encrypted packets back and forth. Instead, the proxy decrypts all incoming information, just like a normal webserver. Then, IF the target is also HTTPS, it re-encrypts traffic it sends onward, but these "proxy-like" aspects of it are invisible and otherwise irrelevant to the client.

Here now is an important consideration. None of the above will work unless the client trusts the key and cert provided above. This is nothing to do with proxying, it's just how public key infrastructure works when a client visits a server (and remember, the proxy is just a server from the client's POV). So one of two things needs to happen:

  1. The above key and cert need to be signed by a known, trusted certificate authority, which your client automatically trusts but which requires giving somebody money.
  2. You must create your own self-signed certificate authority, manually add it to your client's list of trusted authorities, and use it to sign the above key and cert.

With a signed, trusted cert, everything should now work. It's important to realize that there's no spoofing involved here. Yes, content sent to the client is essentially mirrored from another site, but from the client's POV it isn't visiting the that other site in the first place. The URL in the location field says "https://localhost:8080" The key and cert provided are owned by you, and legitimately represent 'localhost'. Everything is kosher.

So how does Hoxy figure into this?

Hoxy follows the above patterns, but with one major deviation having to do with HTTPS CONNECT tunneling. Remember the part above where the proxy receives a CONNECT request and faithfully connects the client to the server, then subsequently shovels encrypted packets back and forth? Hoxy plays a devious prank here. Instead of connecting the client to the server, Hoxy connects the client to its own private HTTPS server, which is a separate instance from the proxy itself and solely exists to spoof TLS. So far, from the client's POV, things are indistinguishable from a normal proxy.

In a normal scenario, the client would now immediately realize something fishy is going on, because that private server, being HTTPS, must provide a key and cert to the client. And even though that cert contains a CN value matching 'localhost', it hasn't been signed by a CA the client trusts. This is why if you want to use Hoxy as an HTTPS direct proxy, you must create your own self-signed root certificate authority, then configure your client to trust it, then pass that CA to Hoxy.

With this in mind, let's rewind to the point just before the client realizes something fishy is going on. Hoxy, having in its possession both the cert AND the private key of a client-trusted CA, will use it to generate spoofed keys and certs, signed by that CA, for each HTTPS domain the client visits. From the client's POV now, nothing fishy is going on. As far as it knows, its CONNECT requests are being honored by the proxy, and it's talking directly to the remote website in a private channel.

But in reality, the client is tunneling to an imposter server, traffic is being decrypted and snooped/intercepted and all the things Hoxy does, before being re-encrypted and sent onward. Neither the client nor the server are the wiser.

Difference between tls and certAuthority options

So having reviewed all this, it should make sense why launching Hoxy as a reverse HTTPS proxy requires a tls key/cert option, whereas launching as a direct HTTPS proxy requires a certAuthority key/cert option. The former is just the normal Node.js setup for running an HTTPS webserver, since a reverse proxy is just a normal webserver, while the latter is a meta-key/cert combo, used to spoof actual key/cert combos on the fly. Also, it's important to realize that these two options are mutually exclusive. It wouldn't make sense to use both at the same time.

Personally, what I've done is to create a self-signed root CA, make all my clients trust it, then save it in a secure location on my system. That way I only have to do this step once. Then, to launch a reverse proxy, I generate a key/cert for it using that root CA, and pass it to Hoxy as tls. If I want to launch a direct proxy, I pass the CA directly to Hoxy as certAuthority. Hypothetically, if I wanted to launch Hoxy as a reverse HTTPS proxy on a public server, I's eschew my self-signed CA and simply pay for a key/cert, similar to how I'd do it for any public HTTPS server.

Generate a self-signed root CA

# Generate the private key for your CA
openssl genrsa -out ~/.ssh/my-private-root-ca.key.pem 2048
# Generate your cert and self-sign it. Details can be bogus.
openssl req -x509 -new -nodes -key ~/.ssh/my-private-root-ca.key.pem -days 1024 -out ~/.ssh/my-private-root-ca.crt.pem -subj "/C=US/ST=Utah/L=Provo/O=ACME Signing Authority Inc/CN=example.com"

Generate a key/cert for an HTTPS webserver

# Generate the private key
openssl genrsa -out ./my-server.key.pem 2048
# Create a CSR
openssl req -new -key ./my-server.key.pem -out ./my-server.csr.pem -subj "/C=US/ST=Utah/L=Provo/O=ACME Tech Inc/CN=localhost"
# Sign the CSR, create a cert
openssl x509 -req -in ./my-server.csr.pem -CA ~/.ssh/my-private-root-ca.crt.pem -CAkey ~/.ssh/my-private-root-ca.key.pem -CAcreateserial -out ./my-server.crt.pem -days 500

Launch a direct HTTPS proxy

new hoxy.Proxy({
  certAuthority: {
    key: fs.readFileSync('/Users/me/.ssh/my-private-root-ca.key.pem'),
    cert: fs.readFileSync('/Users/me/.ssh/my-private-root-ca.cert.pem'),
  }
})

Launch a reverse HTTPS proxy

new hoxy.Proxy({
  reverse: 'https://example.com',
  tls: {
    key: fs.readFileSync(__dirname + '/my-server.key.pem'),
    cert: fs.readFileSync(__dirname + '/my-server.cert.pem'),
  }
})

(credit: https://github.com/coolaj86/node-ssl-root-cas/wiki/Painless-Self-Signed-Certificates-in-node.js)

@snoj
Copy link

snoj commented Jul 24, 2015

@sholladay to answer the question on the difference between the certAuthority and tls options.

certAuthority would be a certificate authority that is trusted by the system which allows the forward https proxy to generate certificates on the fly. For instance, visit https://facebook.com and a facebook.com certificate is generated. If the page needs to connect to westcoast.us.cdn.facebook.com hoxy will generate that too.

The tls option is your usual certificate (and other nodejs https.createServer options) and is used for setting up the https server for a reverse https proxy.

I don't think there would be an instance where both are needed at the same time for the same proxy. Perhaps @greim can verify, but I think the reasoning behind having the two different options is to avoid confusion for the two different purposes.

http://greim.github.io/hoxy/#new-proxy

Just saw @greim's write up. 👍

@greim
Copy link
Owner Author

greim commented Jul 24, 2015

Edit to above: Added openssl commands for generating all keys/certs in question.

@sholladay
Copy link
Contributor

That is fantastic. Thanks, @greim and @snoj, that solves my nitpick. I got direct proxying working pretty quickly - although Chrome warned me about "outdated security" when using it, so probably those openssl commands could use some tweaking. Does anyone else see that?

On a tangential note, I'm curious if there would be any interest in some kind of Yeoman or Inquirer-based setup script to make this even easier. Such a thing may already exist and could get a link in the docs.

@greim
Copy link
Owner Author

greim commented Jul 25, 2015

I hadn't noticed outdated security warnings myself, but maybe I wasn't looking closely enough at the lock icon. I wonder if the "500 days" is too long. Or maybe an outdated algo? Need to do more research.

Hoxy has a simplistic scaffolding CLI, but it doesn't do any cert stuff and I want to remove it in 3.0 in any case. Ideally, someone else could publish a CLI utility on npm based on Hoxy that would include some cert helper/setup stuff.

@sholladay
Copy link
Contributor

Thanks again for all of the hard work on this, everyone. The bounty has been paid to @snoj, I will post another one against #44 to help facilitate some kind of split. I'm afraid the Bountysource tools are not what they ought to be on my end.

@snoj
Copy link

snoj commented Aug 10, 2015

@Phoenixmatrix and @greim, Want to send me your paypal addresses somehow and I'll send over your share when I get it?

If over email, to verify it's really you, send in the email a "secret message" then after a couple days, post that in this thread. Would that work?

@Phoenixmatrix
Copy link

@snoj I just saw this, months later :) (My email filters are a total trainwreck). I have no idea what you ended up doing, but I obviously forfeit anything :)

Thanks though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants