HTTP(S) proxy that saves traffic to a WARC file, using libmitmproxy.
Python
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
hanzo
.gitignore
COPYRIGHT.md
README.md
ca.crt
ca.key
ca.pem
cert.p12
warcmitmproxy.py
warcrecords.py

README.md

WarcMITMProxy

WarcMITMProxy is an HTTP(S) proxy that saves all web traffic to a file. The file format used is the Web ARChive (WARC) format (ISO 28500). WarcMITMProxy uses mitmproxy. This lets you browse the web using a regular browser, and anything you view in the browser is archived in the WARC file.

Prerequisites

Usage

To use WarcMITMProxy, open a web browser and change the proxy settings to connect to 127.0.0.1:8000. Then from a console, run python warcmitmproxy.py. Any browsing activity is passed through the proxy and saved into a WARC file. I recommend browsing with QupZilla, which is a portable, lightweight browser based WebKit.

SSL Certificate

WarcMITMProxy requires an SSL certificate support reading HTTPS sessions. Although the program comes with a pre-generated certificate that can be imported into Windows, the certificate is publicly available, so a personal certificate should be generated by following the steps below.

Generating an SSL Certificate on Windows 7

Install OpenSSL, then use the following commands. They will create four files: ca.key, ca.crt, ca.pem, and ca.p12.

cd C:\OpenSSL-Win32\bin
set OPENSSL_CONF=C:\OpenSSL-Win32\bin\openssl.cfg
openssl genrsa -out ca.key 2048
openssl req -new -x509 -days 365 -key ca.key -out ca.crt
openssl pkcs12 -export -out cert.p12 -inkey ca.key -in ca.crt
copy /B ca.key+ca.crt ca.pem

Using an SSL Certificate

After creating the files, the certificate must be imported into the Windows Certificate manager. Click start and type certmgr.msc and open the application. Right-click the Trusted Root Certification Authorities folder and go to All Tasks > Import. Click Next and then Browse for a certificate. In the filter in the bottom right, change it to "Personal Information Exchange (*.pfx, *.p12)" and import cert.p12.

To use the certificate with WarcMITMProxy, copy cert.pem into WarcMITMProxy's folder.