Proxy Authentication headers missing from HTTPS requests

[flippyhead]

Hello! I've attempted your suggested approach to implement basic authentication (included below). However I'm seeing that HTTPS requests do not include the same header information that HTTP requests include. For example, this HTTP request includes the expected headers:

The CURL request:

curl http://stackoverflow.com -v -x http://username:password@myproxy.com:8089

* Rebuilt URL to: http://stackoverflow.com/
*   Trying 127.0.0.1...
* Connected to myproxy.com (127.0.0.1) port 8089 (#0)
* Proxy auth using Basic with user 'username'
> GET http://stackoverflow.com/ HTTP/1.1
> Host: stackoverflow.com
> Proxy-Authorization: Basic cGV0ZXI6cGFzc3dvcmQ=
> User-Agent: curl/7.43.0
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< date: Sat, 12 Mar 2016 16:03:29 GMT
< content-type: text/html; charset=utf-8
< transfer-encoding: chunked
< connection: close
< set-cookie: __cfduid=d0a207fda936835992317de3be8a24ecc1457798609; expires=Sun, 12-Mar-17 16:03:29 GMT; path=/; domain=.stackoverflow.com; HttpOnly
< set-cookie: prov=359b8d31-d451-4ea5-8a12-8824bb92b971; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
< cache-control: public, no-cache="Set-Cookie", max-age=60
< expires: Sat, 12 Mar 2016 16:04:29 GMT
< last-modified: Sat, 12 Mar 2016 16:03:29 GMT
< vary: *
< x-frame-options: SAMEORIGIN
< x-request-guid: 91d04c64-9c57-4475-a188-e917ec63193d
< server: cloudflare-nginx
< cf-ray: 282879fbf28c2a5b-SEA
< 

The node-http-mite-proxy request headers:

{ 
  host: 'stackoverflow.com',
  'proxy-authorization': 'Basic cGV0ZXI6cGFzc3dvcmQ=',
  'user-agent': 'curl/7.43.0',
  accept: '*/*',
  'proxy-connection': 'Keep-Alive' 
}

Whereas an HTTPS request, while showing similar header information in the CURL request, does not include the same headers in the proxy:

The CURL request

curl https://github.com -v -x http://username:password@myproxy.com:8089

* Rebuilt URL to: https://github.com/
*   Trying 127.0.0.1...
* Connected to my proxy.com (127.0.0.1) port 8089 (#0)
* Establish HTTP proxy tunnel to github.com:443
* Proxy auth using Basic with user 'username'
> CONNECT github.com:443 HTTP/1.1
> Host: github.com:443
> Proxy-Authorization: Basic cGV0ZXI6cGFzc3dvcmQ=
> User-Agent: curl/7.43.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< 
* Proxy replied OK to CONNECT request
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: github.com
* Server certificate: NodeMITMProxyCA
> GET / HTTP/1.1
> Host: github.com
> User-Agent: curl/7.43.0
> Accept: */*
> 
< HTTP/1.1 407 Proxy Authentication Required
* Authentication problem. Ignoring this.
< Proxy-Authenticate: Basic
< Date: Sat, 12 Mar 2016 16:06:34 GMT
< Connection: keep-alive
< Transfer-Encoding: chunked
< 
* Connection #0 to host local.fetching.io left intact

The headers included in the proxy:

{ 
  host: 'github.com',
  'user-agent': 'curl/7.43.0',
  accept: '*/*' 
}

My proxy code:

'use strict';

const Proxy = require('http-mitm-proxy');
const proxy = Proxy();
const logger = require('./logger.js');

proxy.onRequest( (ctx, callback) => {
  let request = ctx.clientToProxyRequest;
  let host = request.headers['host'];
  let path = request.url;
  let protocol = ctx.isSSL ? 'https' : 'http';
  let url = `${protocol}://${host}${path}`;
  let proxyAuthorization = request.headers['proxy-authorization'];

  console.log('DEBUG', url, request.headers);

  if (proxyAuthorization) {
    let encodedLoginPassword = proxyAuthorization.replace('Basic ', '');
    let loginPassword = new Buffer(encodedLoginPassword, 'base64').toString().split(':');
    let login = loginPassword[0];
    let password = loginPassword[1];
    callback();
  } else {
    let response = ctx.proxyToClientResponse;
    response.writeHead(407, {'Proxy-Authenticate': 'Basic'});
    response.end();
  }
});

proxy.onRequest( (ctx, callback) => {
  let request = ctx.clientToProxyRequest;
  let proxyAuthorization = request.headers['proxy-authorization'];
  callback();
});

proxy.onError( (ctx, err, errorKind) => {
  let url = (ctx && ctx.clientToProxyRequest) ?
    ctx.clientToProxyRequest.url :
    '';
  logger.log('error', `${errorKind} on ${url}`, err);
});

proxy.listen({
  port: 8089,
  forceSNI: true,
  silent: true,
  sslCaDir: "~/Development/proxy/certs"
});

In short, the proxy authentication works for HTTP requests but doesn't work for HTTPS requests and I'm not sure what accounts for the difference.

Thank you!

[felicienfrancois]

Here what's happening:
HTTP: the client send directly the full request to the proxy, with the proxy-auth headers. The proxy is in charge to forward to server.
HTTPS: the client want to send a request to a server, encrypted with the server public key, passing through an http proxy. So

• before making the request itself, the client have to get the server public key (i.e. make SSL handshake, i.e. etablish a SSL/TLS tunnel to the server)
• the HTTPS request will be encrypted so the proxy won't have access to the request headers (in case of a normal proxy)
  What happens:
• The client send a HTTP CONNECT to the proxy, indicating the hostname of the server to connect with and also probably the proxy-auth headers
• The proxy connect the client with the server (here with an internal https server instead of remote one, to be able to decrypt and modify)
• Client and server (internal one here) makes SSL handshake
• Client send the HTTPS request to the server, without proxy-auth headers because the proxy is not supposed to have access to it

So in order to access the proxy-* headers on HTTPS, you may have to hook into the connect process

[avistramer]

#171 should have solved this one. Here is my code now if it helps others:

proxy.onConnect(function(req, socket, head, callback) {
	if (!req.headers['proxy-authorization']) {
		socket.write('HTTP/1.1 407 Proxy Authentication Required\r\n');
		socket.write('Proxy-Authenticate: Basic realm=\"My realm\"\r\n');
		socket.write('Proxy-Connection: keep-alive\r\n');
		socket.write('Connection: keep-alive\r\n');
        	socket.write('\r\n');
        	return;
	}
	// validate req.headers['proxy-authorization'] here
	// ......
	callback(); 
});

proxy.onRequest(function(ctx, callback) {
	if (!ctx.isSSL && !ctx.clientToProxyRequest.headers['proxy-authorization']) {
		ctx.proxyToClientResponse.setHeader('Proxy-Authenticate', 'Basic realm=\"My realm\"');
		ctx.proxyToClientResponse.statusCode = 407;
		ctx.proxyToClientResponse.statusMessage = 'Proxy Authentication Required';
		ctx.proxyToClientResponse.end();
        	return;
	}
	var authHeader = ctx.clientToProxyRequest.headers['proxy-authorization'] || ctx.connectHeaders['proxy-authorization'];
	// validate authHeader here
	// ....
	callback();
});

proxy.onWebSocketConnection(function(ctx, callback) {
	var authHeader = ctx.connectHeaders['proxy-authorization'];
	// validate authHeader here
	// ....
	callback();
});