Support HTTPS

[kdzwinel]

HTTPS request are not logged by the proxy. At this point I have no good idea how to approach this - research needed.

Question: will it be possible to decode body of these requests? This will probably break connection for apps being debugged? Or do self-signed certs fix that? Can I create them on flight?

[benwa]

Using mitmproxy would achieve what you're looking for. I know Postman uses it (though never got it to work).
It does require a cert to be installed on the device.

[janpio]

Look at what Charles Web Proxy does. This process is quite slick and works like a charm.

[lvauvillier]

Here is a nodejs project that implements HTTPS interception: https://github.com/axiak/filternet
BTW I really like his motivation: "I created and used this module to alter my now wife's Google doodle to eventually lead to a proposal. She said yes, so this project was a success."

[ivanistheone]

+1 for https support. Presumably, a certificate warning will show up, but that's fine.

[kdzwinel]

Tried a few things and https://github.com/joeferner/node-http-mitm-proxy seems to be the best match. Work in progress.

[kdzwinel]

First working version: #31 .

Tips for working with self-signed certificates:

• setting https proxy in the terminal: export https_proxy=http://localhost:8008
• running wget without certificate checking: wget --no-check-certificate https://google.com
• running chrome without certificate checking: /Applications/Google\ Chrome\ Canary.app/Contents/MacOS/Google\ Chrome\ Canary --ignore-urlfetcher-cert-requests --ignore-certificate-errors --user-data-dir="/tmp/chrome_tmp_profile" --proxy-server="http://localhost:8008"

BTW Does anyone know if there is a way to make all certificates created by the node-http-mitm-proxy valid system wide? Maybe, somehow register it as a valid certificate authority?

[kdzwinel]

Answering question from my previous comment:

Setting NodeMITMProxyCA as a valid Certificate Authority is super straightforward (at least on OSX) and works great for most apps (some apps do not respect global proxy settings - e.g. Dropbox).

Steps:

• run Betwixt (to generate the certificate)
• go to the ~/Library/Application Support/betwixt/ssl/certs folder that is created by Betwixt on the first run (OSX specific, for other OSes see: getPath(appData)).
• import ca.pem using Keychain and mark it as trusted

Set HTTP_S_ proxy in the Settings > Network > Advanced > Proxies > Secure Web Proxy (HTTPS) to localhost:8008 (as you normally do with HTTP) and enjoy:

[Bellardia]

I've tried to build from source, but the http-mitm-proxy directory is not created, nor can I find the contents anywhere else.

[kdzwinel]

@Bellardia thanks for reporting, I'll try to reproduce that - it's osx, isn't it?

[Bellardia]

@kdzwinel Yes it is.
I was able to get it working, but it took a bit of manual work.
I built the http-mitm-proxy directly, copied the .http-mitm-proxy directory from that project to the root of Betwixt.
I then had to override the path in Betwixt, since it seems to be using absolute pathing?
line 24 of traffic-interceptor.js:
proxy.listen({port: port, sslCaDir: ''});

[TACIXAT]

I just want to add a note for the SSLKEYLOGFILE. Both FF and Chrome will log session keys to the file specified in that environment variable. That would circumvent adding a cert and get around sites that do cert pinning (applications too, as long as you can get them to log their session keys, but that should be possible via patching the app).

[kdzwinel]

@Bellardia I was able to reproduce it - TLDR mitm-proxy uses process.cwd() that behaves a bit unexpectedly inside a packaged app. I used electron specific function instead - now certs and keys will be available in ~/Library/Application Support/betwixt/ssl/certs folder on OSX (for other OSes see: http://electron.atom.io/docs/v0.37.2/api/app/#appgetpathname ). I'm trying to figure a way to expose ca.pem in the Betwixt UI.

[kdzwinel]

HTTPS support landed.

New binaries: https://github.com/kdzwinel/betwixt/releases/tag/1.0.0
HTTPS docs: https://github.com/kdzwinel/betwixt/blob/master/docs/https.md

Please let me know about any issues.

[sagivf]

Hey there, thanks for this cool tool 😄
I'm trying yo use this with https on ubuntu.
Do you have any info on setting up the certificate?

I get the following error:

This is my network setup:

[kdzwinel]

@sagivf
Betwixt generates a certificate for you: https://github.com/kdzwinel/betwixt/blob/master/docs/https.md#root-certificate

What you have to do is install it. Haven't tried it myself, but it seems to be simple enough: https://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate

[sagivf]

Hey @kdzwinel I found the cs.pem and followed the instructions:

• moved to /usr/share/ca-certificates/extra/
• changed to .crt
• added via sudo dpkg-reconfigure ca-certificates

Im still getting the above error. Any idea what else I can do?

[barneycarroll]

@sagivf I followed the instructions for OSX but needed the additional step of telling my keychain manager to trust the certificate (which it didn't according to defaults).

[sagivf]

Thanks @barneycarroll that will be helpful when for my IOS development, unfortunately I mostly use ubuntu and cant get it to work...

[barneycarroll]

@sagivf I'm not on Ubuntu but assuming you have the same underlying issue, you would have to take some kind of action to ensure the certificate is recognised as trusted. I haven't tried it but this looks promising…