Frankencert - Adversarial Testing of Certificate Validation in SSL/TLS Implementations
What are frankencerts?
Frankencerts are specially crafted SSL certificates for testing certificate validation code in SSL/TLS implementations. The technique is described in detail in the 2014 IEEE Symposium on Security and Privacy (Oakland) paper - Using Frankencerts for Automated Adversarial Testing of Certificate Validation in SSL/TLS Implementations by Chad Brubaker, Suman Jana, Baishakhi Ray, Sarfraz Khurshid, and Vitaly Shmatikov.
Why is frankencert generator useful?
Frankencert generator is essentially a smart fuzzer for testing SSL/TLS certificate validation code. If you are a developer who is implementing any sort of SSL/TLS certificate validation code (either as part of an SSL/TLS library or an application), you can use the frankencert generator to auto-generate different test certificates involving complex corner cases.
We have successfully used frankencerts to find serious vulnerabilities in GnuTLS, PolarSSL, CyaSSL, and MatrixSSL as described in our Oakland 2014 paper. We also found several discrepancies between how different SSL/TLS implementations report errors back to the user. For example, when presented with an expired, self-signed certificate, NSS, Chrome on Linux, and Safari report that the certificate has expired but not that the issuer is invalid.
How do frankencerts work?
The basic idea of frankencerts is to take a bunch of certificates as seeds and use random mutations on different fields and extensions to create new test certificates (frankencerts). Using frankencerts as server-side inputs into an SSL/TLS handshake can help systematically test correctness of the certificate validation code.
Installation and Usage
Install OpenSSL libraries and utilities if you don't have them already. On Ubuntu this can be performed with
sudo apt-get install libssl-dev.
Install the python3-setuptools package used by PyOpenSSL with
sudo apt-get install python3-setuptools.
The frankencert generator needs a modified version of PyOpenSSL. We have included the source for our modified version of PyOpenSSL. You will need to install it in order to use the frankencert generator. First, uninstall any other version of PyOpenSSL that you may have installed on your computer. Go to the
pyopenssl-19.1.0directory and build/install PyOpenSSL by issuing
sudo python3 setup.py install.
Once you have the patched pyOpenSSL set up, to generate frankencerts, use the
python3 franken_generate.py seed_certs_dir ca_cert output_dir count [config_file]. Example usage:
python3 franken_generate.py ../utils/test_certs/ ../utils/rootCA_key_cert.pem ../output/ 30.
The arguments are explained below.
seed_certs_dir: Frankencert generator needs a set of seed certificates. Any SSL cert in PEM fromat can act as a seed cert.
seed_certs_dircan be any directory containing the seed certs stored as PEM files.
You can either use tools like ZMap (https://zmap.io/) to collect SSL seed certificates, or use some of the SSL certs available from https://www.eff.org/observatory. Please note that these are not our tools and repositories - you may want to contact their respective developers and maintainers to ensure that your usage of the certificates they collected is compatible with the intended purpose. You do not need access to the corresponding private keys to use the certs as seeds.
For your convenience, we have included a tarball containing around 1000 seed certificates in
ca_cert: You will also need to create a self-signed CA certificate to sign the frankencerts. You can either use the included sample CA certificate
utils/rootCA_key_cert.pemor use the
utils/create_new_ca.shscript to create your own root CA. For any root CA that you use for frankencert generation, make sure that your SSL certificate validation code treats its certificate as a trusted root certificate.
VERY IMPORTANT: this root certificate should be trusted ONLY during testing. If you accidentally or intentionally deploy SSL/TLS with this certificate still among the trusted root certificates, your SSL/TLS connections may be vulnerable to server impersonation and man-in-the-middle attacks. Be sure to REMOVE this certificate from your trusted root certificates once the testing is finished.
output_dir: It will contain the generated frankencerts. The frankencerts will be named as
count: Number of frankencerts to be generated.
config_file: An optional argument to tune the frankencert generation process. Take a look at the
utils/sample_franken.conffor a sample config file.
To test your SSL/TLS client with the generated frankencerts, you should use the
utils/test_ssl_server.pyscript to set up an SSL server that can present the generated frankencerts as part of the SSL handshake. Example usage:
python3 test_ssl_server.py ../output/frankencert-0.pem 443
If you want to perform differential testing (i.e., compare your SSL/TLS client's behavior with other libraries' behaviors for a given frankencert), you can do so by running SSL clients using those libraries (some samples are available in
utils/differential_testing_sample_clients.tar.gz) and connecting to a server serving the frankencert. The following example shows how to do this for OpenSSL.
- Start a SSL server serving the target frankencert using:
./test_ssl_server.py frankencert_name port_no, where
frankencert_nameis the path of the target frankencert and
port_nois the port the server will listen to.
- Use the command
openssl s_client -CAfile ca_cert -connect host_name:port_noto connect to the server and check the certificate verification result printed on the console. The
ca_certargument should be the CA certificate you used to generate the frankencert,
port_noshould be the same one that you used for running
host_nameshould be either localhost or the name of the host running the
- Start a SSL server serving the target frankencert using:
frankengendirectory contains the frankencert generator code
- Our patched version of pyOpenSSL is inside
- Several useful tools are included in
cert_print.py: a tool for printing frankencerts. It requires OpenSSL to be installed and present in the path.
rootCA_key_cert.pem: private key and self-signed cert of a sample CA that can be used for signing frankencerts.
create_new_ca.sh: a script for creating new CA with a self-signed cert. It creates the output cert and private key in
test_ssl_server.py: a sample SSL/TLS server for presenting frankencerts to SSL/TLS clients
sample_seed_certs.tar.gz: Some sample certs that may be used as seeds for frankencert generation.
sample_franken.conf: A sample config file that can be used to tune different parameters of the frankencert generation process.
differential_testing_sample_clients.tar.gz: C code for SSL/TLS clients using different SSL libraries that connect to a SSL server. The error codes (if any) are printed on the console. These clients can be used to perform differential testing.
initialize_differential_testing.sh: A script for pulling and building openssl-1.0.1e and gnutls-220.127.116.11, and compiling the sample openssl and gnutls clients with the specific libraries for initializing a differential testing environment.