WarcMITMProxy is an HTTP(S) proxy that saves all web traffic to a file. The file format used is the Web ARChive (WARC) format (ISO 28500). WarcMITMProxy uses mitmproxy. This lets you browse the web using a regular browser, and anything you view in the browser is archived in the WARC file.
pip install mitmproxy
To use WarcMITMProxy, open a web browser and change the proxy settings to
connect to 127.0.0.1:8000. Then from a console, run
Any browsing activity is passed through the proxy and saved into a WARC file.
I recommend browsing with QupZilla, which is a
portable, lightweight browser based WebKit.
WarcMITMProxy requires an SSL certificate support reading HTTPS sessions. Although the program comes with a pre-generated certificate that can be imported into Windows, the certificate is publicly available, so a personal certificate should be generated by following the steps below.
Generating an SSL Certificate on Windows 7
Install OpenSSL, then use the following commands. They will create four files: ca.key, ca.crt, ca.pem, and ca.p12.
cd C:\OpenSSL-Win32\bin set OPENSSL_CONF=C:\OpenSSL-Win32\bin\openssl.cfg openssl genrsa -out ca.key 2048 openssl req -new -x509 -days 365 -key ca.key -out ca.crt openssl pkcs12 -export -out cert.p12 -inkey ca.key -in ca.crt copy /B ca.key+ca.crt ca.pem
Using an SSL Certificate
After creating the files, the certificate must be imported into the Windows Certificate manager. Click start and type certmgr.msc and open the application. Right-click the Trusted Root Certification Authorities folder and go to All Tasks > Import. Click Next and then Browse for a certificate. In the filter in the bottom right, change it to "Personal Information Exchange (*.pfx, *.p12)" and import cert.p12.
To use the certificate with WarcMITMProxy, copy cert.pem into WarcMITMProxy's folder.