The goal of this project is to allow for accurate detection of HTTPS interception and robust TLS fingerprinting. This project is based off of The Security Impact of HTTPS Interception, and started as a port to Go of their processing scripts and fingerprints.
Signature and Fingerprints
In this project, fingerprints map to concrete instantiations of an object, while signatures can represent multiple objects. We use this convention because a fingerprint is usually an inherent property of an object, while a signature can be chosen. In the same way, an actual client request seen by a server would have a fingerprint, while the software generating the request can choose it's own signature (e.g., by choosing which cipher suites it supports).
A client request fingerprint is derived from a client request to a server, and contains both TLS and HTTP features. A client request signature represents all of the possible fingerprints that a piece of software can generate. The aim is to make each signature specific enough that it can uniquely identify a piece of software.
A user agent signature represent a set of user agents generated by a browser. A user agent signature for a browser allows for a range of browser versions, and allows for specifying the OS name, OS platform, OS version range, and device type for creating more fine-grained signatures.
A browser signature contains both a user agent signature and a client request signature. This allows for a signature to represent all of the possible fingerprints generated by Chrome 31-38 on Windows 10, for example.
A MITM signature contains a client request signature along with additional details about the MITM software, including a security grade which can be affected by factors outside of the client request, such as whether or not the software validates certificates.
MITM Detection Methodology
We consider an HTTPS connection to be intercepted when there is a mismatch between the expected client request signature corresponding to the browser identified by the user agent, and the actual client request fingerprint of the request.
If a signature is inaccurate or outdated for a given piece of client software, it is possible that the signature will falsely flag a connection as being intercepted.
If a proxy closely mimics the request of the client, then we may not expect to detect a mismatch. If the browser signatures are overly broad, we will also fail to detect interception.
To test, run
make test and to see code coverage, run
make godoc or
PKG=<sub-package> make godoc to generate godoc for mitmengine or any of the sub-packages.
First, a user must create a
mitmengine.Config struct to pass into
struct can either specify filenames of files containing browser fingerprints, MITM fingerprints, and man-in-the-middle
headers. Alternatively, it can also specify a configuration file for reading the previously mentioned files from any
other source; right now, mitmengine supports reading these files from Amazon S3 client-compatible databases (including
Amazon S3 and Ceph). Additional file readers for databases (which we call "loaders") can be defined in the
package, and as long as new loaders implement the Loader interface, they should work with the rest of mitmengine out of the
box. We added support for additional fingerprint and bad header sources in the case mitmengine is run as a daemon and
you want to have it periodically update the fingerprint and bad header files it uses to analyze traffic.
The intended entrypoint to the mitmengine package is through the
Processor.Check function, which takes a user agent and client request fingerprint, and returns a mitm detection report. Additional API functions will be added in the future to allow for adding new signatures to a running process, for example.