This testbed aims to test the SPHINCS+ framework inside the TLS context.
First, SPHINCS+ along with all its variants is tested and compared to standard signature schemes: RSA and ECDSA. This is done exploiting the liboqs testsuite library.
Second, SPHINCS+ is used as the signature algorithm in the authentication phase of TLS handshake procedure. The performance of TLS handshakes made with SPHINCS+ are compared to TLS handshakes performed with standard signature algorithms.
In the last decades, post-quantum cryptography has become an active area of research. In 2016, NIST started the post-quantum crypto project in order to encourage the development of new post-quantum signature and key exchange algorithms. SPHINCS+ is one of the post-quantum signature schemes proposed to NIST. Since it is a new cryptographic algorithm, it has not been fully tested yet. Moreover, just few works tested SPHINCS+ in a real-world use case scenario.
In this project we want to test SPHINCS+ inside the TLS protocol, in order to provide some results about the overhead introduced by this post-quantum signature algorithm with respect to standard cryptography algorithms.
The testbed is divided into 2 main parts:
- liboqs: the scripts
build_testbed_liboqs.sh,run_testbed_liboqs.sh,run_testbed_liboqs_rsaecdsa.shandclean_testbed_liboqs.shfocus on the performance evaluation of SPHINCS+. For completeness, also the most common post-quantum key exchange mechanisms are evaluated. - openssl: the scripts
build_testbed.sh,build_testbed_single_variant.sh,run_testbed.sh,run_testbed_rsaecdsa.shandclean_testbed.shfocus on the performance evaluation of SPHINCS+ inside the OpenSSL library.
See the How to use section to have a description of the specific functionalities provided by each script.
In this section we are going to describe the goal of each .sh script. All outputs provided by this testbed are organized inside the results directory, contained inside the tests directory which contains all scripts. They will be described in the next section.
With this part both liboqs and openssl libraries are installed. Also the latter is installed because standard cryptographic algorithms can not be tested through the liboqs testsuite, so the openssl testsuite need to be used (for RSA and ECDSA).
build_testbed_liboqs.sh: downloads both liboqs and openssl libraries inside the testbed_for_liboqs_tests directory, and compiles them. The liboqs is compiled to have as default signature algorithm SPHINCS+-Haraka-128f-robust. This is used to issue X.509 certificates. In this way if the library needs to be used afterward, it is ready to be tested.run_testbed_liboqs.sh: runs the test suites provided by the liboqs library for post-quantum algorithms. All SPHINCS+ variants are tested with thetest_sigscript inside liboqs. Moreover, post-quantum KEMs like BIKE, Frodo and NTRU are tested with thetest_kemscript inside liboqs. These algorithms are evaluated because after having found the best one, it will be used inside the OpenSSL related tests for evaluating the performance of a TLS handshake with both post-quantum signature and key exchange algorithms.run_testbed_liboqs_rsaecdsa.sh: runs multiple times (10) the openssl speed command for evaluating the performance of RSA2048 and ECDSA384 algorithms. The command is executed multiple times in order to have more precise results, computing the mean and the standard deviation of the obtained values.clean_testbed_liboqs.sh: deletes the testbed_for_liboqs_tests directory.
build_testbed.sh: this is the most important script in the testbed of this SPHINCS+ - TLS project. First it iterates on the list of all SPHINCS+ variants (36). It compiles the liboqs with the current variant as the default signature algorithm. Then both the liboqs and openssl libraries are built. It then calls therun_testbed.shscript passing as parameter the current SPHINCS+ variant. Then, if the memory of the current machine is not enough to support multiple copies of the liboqs and openssl libraries, it calls theclean_testbed.shscript. Those libraries have to be rebuilt for each SPHINCS+ variant since the implementation of SPHINCS+ into OpenSSL is not available yet. The only way to test it into the TLS scenario is to compile the liboqs library with SPHINCS+ as the default signature algorithm.build_testbed_single_variant.sh: support script which enables to build the liboqs and openssl libraries to select as the default signature algorithm the one specificed in the command line as first parameter. The algorithm has to be specified in the form OQS_SIG_alg_sphincs_haraka_128f_robust.run_testbed.sh: this script tests a single SPHINCS+ variant, passed as first parameter by command line. It launches a TLS server and evaluates the performance of the TLS handshake with thes_timecommand, multiple times (10). For each SPHINCS+ variant, the TLS handshake is performed with the Diffie-Hellman key exchange and with the NTRU hps2048509 post-quantum key exchange algorithm. This is the post-quantum key exchange algorithm chosen since it was the most efficient post-quantum KEM among the ones tested by therun_testbed_liboqs.shscript.run_testbed_rsaecdsa.sh: it evaluates the performances of TLS handshakes performed with RSA2048 and ECDSA384 signature algorithms. It can be used to use as target directory the one specified by command line. For each signature algorithm, it creates inside openssl the keys and the X.509 certificates, and then iterates multiple times (10) executing thes_timecommand.clean_testbed.sh: deletes the testbed directory.
The output files produced by this testbed are contained inside the ~/tests/results folder. All files related to the first part of the testbed, the one described in the liboqs section, are in the format output_liboqs_*.txt.
All files related to the second part of the testbed, the one focusing on TLS integration, are in the format output_tls_s_time_*.txt.
output_liboqs_kem.txtcontains the performance values for encapsulating and decapsulating keys for the selected post-quantum KEM algorithms.output_liboqs_sig.txtcontains the performance values for the key generation, sign and verify operations for post-quantum authentication mechanisms (SPHINCS+).output_liboqs_std_sig.txtcontains the performance values for sign and verify operations for standard signature algorithms (RSA2048 and ECDSA384).output_tls_s_time.txtcontains the statistics for TLS handshakes performed with SPHINCS+ as the signature algorithm inside the authentication phase of the handshake.output_tls_s_time_rsaecdsa.txtcontains the statistics for TLS handshakes performed with RSA2048 and ECDSA384 as the signature algorithms inside the authentication phase of the handshake.
For now, the content of these files is the copy of the ouput the exploited test commands of the liboqs and openssl libraries.
In order to use this testbed, download and install it into home directory (~). That is because some scripts use absolute paths to work. In order to change this behaviour the paths inside the scripts could be changed. The path for one of the scripts should be ~/tests/script.sh.
This testbed relies on the OpenSSL library, version 1.1.1 and on the liboqs library.
These libraries are cloned at runtime, installing and building the last available versions.
Future work for this testbed:
- Improve the readability of the output produced by the
run_*.shscripts, in order to produce a .csv output instead of a .txt file, which can be automatically manipulated in order to compute the desired statistics. - Avoid the compilation of liboqs and openssl libraries in order to set manually one of the SPHINCS+ variants as the default signature algorithm. This will be possible when the OpenSSL library will fully implement all SPHINCS+ variants directly. In that scenario, only one compilation of the libraries is necessary in order to test all variants with TLS handshakes.
- Avoid to clone the liboqs and openssl repository at each time, but performing a check of the last available version and download it only if it is different from the already installed one.
- Improve the management of paths, replacing input and output absolute path with relative paths.
This project was developed for the Computer System Security course of Politecnico di Torino, Italy. The code was developed by Giulia Milan, under the supervision of Eng. Ignazio Pedone and Professor Antonio Lioy.
MIT open license.