bench: increase cert chain length (aws#4287)

* increase cert chain length to 3 (root, intermediate, leaf)
* add P-256 certificates
* remove config files

bindings/rust/bench/README.md

@@ -23,21 +23,21 @@ cargo bench --bench handshake --bench throughput -- --profile-time 5
 rm -rf .cargo
 ```
 
-## Setup 
+## Setup
 
-Setup is easy! Just have OpenSSL installed, generate Rust bindings for s2n-tls using `../generate.sh`, and generate certs using `scripts/generate-certs.sh`. 
+Setup is easy! Just have OpenSSL installed, generate Rust bindings for s2n-tls using `../generate.sh`, and generate certs using `scripts/generate-certs.sh`.
 
-Dependencies are the same as with s2n-tls. Currently, this crate has only been tested on Ubuntu (both x86 and ARM), but we expect everything to work with other Unix environments. 
+Dependencies are the same as with s2n-tls. Currently, this crate has only been tested on Ubuntu (both x86 and ARM), but we expect everything to work with other Unix environments.
 
-To bench with AWS-LC, Amazon's custom libcrypto implementation, first run `scripts/install-aws-lc.sh` to install AWS-LC for the bench crate. To then run the benchmarks with AWS-LC, use Cargo with either the flag `--config aws-lc-config/s2n.toml` or `--config aws-lc-config/rustls.toml` (or both). You can also append these configs to `.cargo/config.toml` to let Cargo automatically detect the settings without specifying the flags each time.  
+To bench with AWS-LC, Amazon's custom libcrypto implementation, first run `scripts/install-aws-lc.sh` to install AWS-LC for the bench crate. To then run the benchmarks with AWS-LC, use Cargo with either the flag `--config aws-lc-config/s2n.toml` or `--config aws-lc-config/rustls.toml` (or both). You can also append these configs to `.cargo/config.toml` to let Cargo automatically detect the settings without specifying the flags each time.
 
 ### Features
 
 Default features (`rustls` and `openssl`) can be disabled by running the benches with `--no-default-features`. The non-default `memory` and `historical-perf` features are used to enable dependencies specific to those types of benches, and are automatically used by the scripts that run those benches.
 
 ## Performance benchmarks
 
-The handshake and throughput benchmarks can be run with the `cargo bench` command. Criterion will auto-generate an HTML report in `target/criterion/`. 
+The handshake and throughput benchmarks can be run with the `cargo bench` command. Criterion will auto-generate an HTML report in `target/criterion/`.
 
 Throughput benchmarks measure round-trip throughput with the client and server connections in the same thread for symmetry. In practice, a machine would either host only the client or only the server and use multiple threads, so throughput for a single connection could theoretically be up to ~4x higher than the values from the benchmarks (when run on the same machine).
 
@@ -74,9 +74,24 @@ heaptrack target/release/memory (pair|client|server) (s2n-tls|rustls|openssl)
 
 To do historical benchmarks, run `scripts/bench-past.sh`. This will checkout old versions of s2n-tls back to v1.3.16 in `target/` and run benchmarks on those with the `historical-perf` feature, disabling Rustls and OpenSSL benches.
 
+## PKI Structure
+```
+   ┌────root──────┐
+   │              │
+   │              │
+   ▼              │
+ branch           │
+   │              │
+   │              │
+   │              │
+   ▼              ▼
+ leaf            client
+```
+`generate-certs.sh` will generate 4 certificates for each key type, with the signing relationships that are indicated in the diagram above. This cert chain length was chosen because it matches the cert chain length used by public AWS services.
+
 ### Caveats
 
-The last version benched is v1.3.16, since before that, the s2n-tls Rust bindings have a different API and would thus require a different bench harness to test. 
+The last version benched is v1.3.16, since before that, the s2n-tls Rust bindings have a different API and would thus require a different bench harness to test.
 
 v1.3.30-1.3.37 are not benched because of depedency issues when generating the Rust bindings. However, versions before and after are benched, so the overall trend in performance can still be seen without the data from these versions.
 
@@ -87,7 +102,7 @@ Because these benches take a longer time to generate (>30 min), we include the r
 Notes:
 - Two sets of parameters for the handshake couldn't be benched before 1.3.40, since security policies that negotiated those policies as their top choice did not exist before then.
 - There is no data from 1.3.30 to 1.3.37 because those versions have a dependency issue that cause the Rust bindings not to build. However, there is data before and after that period, so the performance for those versions can be inferred via interpolation.
-- The improvement in throughput in 1.3.28 was most likely caused by the addition of LTO to the default Rust bindings build. 
+- The improvement in throughput in 1.3.28 was most likely caused by the addition of LTO to the default Rust bindings build.
 - Since the benches are run over a long time, noise on the machine can cause variability, and background processes can cause spikes.
 - The variability can be seen with throughput especially because it is calculated as the inverse of time taken.
 
@@ -97,11 +112,11 @@ Notes:
 
 ## Implementation details
 
-We use Rust bindings for s2n-tls and OpenSSL. All of our benchmarks are run in Rust on a single thread for consistency. 
+We use Rust bindings for s2n-tls and OpenSSL. All of our benchmarks are run in Rust on a single thread for consistency.
 
 ### IO
 
-To remove external factors, we use custom IO with our benchmarks, bypassing the networking layer and having the client and server connections transfer data to each other via a local buffer. 
+To remove external factors, we use custom IO with our benchmarks, bypassing the networking layer and having the client and server connections transfer data to each other via a local buffer.
 
 ### Certificate generation
 

bindings/rust/bench/benches/resumption.rs

@@ -67,7 +67,7 @@ where
 /// s2n-tls?".
 pub fn bench_resumption(c: &mut Criterion) {
     // compare resumption savings across both client and server
-    for sig_type in [SigType::Rsa2048, SigType::Ecdsa384] {
+    for sig_type in [SigType::Rsa2048, SigType::Ecdsa256] {
         let mut bench_group = c.benchmark_group(format!("resumption-pair-{:?}", sig_type));
         bench_handshake_pair::<S2NConnection>(&mut bench_group, sig_type);
     }

bindings/rust/bench/scripts/generate-certs.sh

@@ -35,40 +35,108 @@ cert-gen () {
     mkdir -p $dir_name
     cd $dir_name
 
+    # The "basicConstraints" and "keyUsage" extensions are necessary for CA
+    # certificates that sign other certificates. Normally the openssl x509 tool
+    # will ignore the extensions requests in the .csr, but by using the
+    # copy_extensions=copyall flag we can pass the extensions from the .csr on
+    # to the final public certificate.
+
+    # The advantage of manually specifying the extensions is that there is no
+    # dependency on any openssl config files
+
     echo "generating CA private key and certificate"
-    openssl req -new -nodes -x509 -newkey $key_family -pkeyopt $argname$key_size -keyout  ca-key.pem -out ca-cert.pem -days 65536 -config ../config/ca.cnf
+    openssl req -new -noenc -x509 \
+            -newkey $key_family \
+            -pkeyopt $argname$key_size \
+            -keyout  ca-key.pem \
+            -out ca-cert.pem \
+            -days 65536 \
+            -subj "/C=US/CN=root" \
+            -addext "basicConstraints = critical,CA:true" \
+            -addext "keyUsage = critical,keyCertSign"
+
+    echo "generating intermediate private key and CSR"
+    openssl req  -new -noenc \
+            -newkey $key_family \
+            -pkeyopt $argname$key_size \
+            -keyout intermediate-key.pem \
+            -out intermediate.csr \
+            -subj "/C=US/CN=branch" \
+            -addext "basicConstraints = critical,CA:true" \
+            -addext "keyUsage = critical,keyCertSign"
 
     echo "generating server private key and CSR"
-    openssl req  -new -nodes -newkey $key_family -pkeyopt $argname$key_size -keyout server-key.pem -out server.csr -config ../config/server.cnf
+    openssl req  -new -noenc \
+            -newkey $key_family \
+            -pkeyopt $argname$key_size \
+            -keyout server-key.pem \
+            -out server.csr \
+            -subj "/C=US/CN=leaf" \
+            -addext "subjectAltName = DNS:localhost"
 
     echo "generating client private key and CSR"
-    openssl req  -new -nodes -newkey $key_family -pkeyopt $argname$key_size -keyout client-key.pem -out client.csr -config ../config/client.cnf
+    openssl req  -new -noenc \
+            -newkey $key_family \
+            -pkeyopt $argname$key_size \
+            -keyout client-key.pem \
+            -out client.csr \
+            -subj "/C=US/CN=client" \
+            -addext "subjectAltName = DNS:localhost"
+
+    echo "generating intermediate certificate and signing it"
+    openssl x509 -days 65536 \
+            -req -in intermediate.csr \
+            -CA ca-cert.pem \
+            -CAkey ca-key.pem \
+            -CAcreateserial \
+            -out intermediate-cert.pem \
+            -copy_extensions=copyall
 
     echo "generating server certificate and signing it"
-    openssl x509 -days 65536 -req -in server.csr -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extensions req_ext -extfile ../config/server.cnf
+    openssl x509 -days 65536 \
+            -req -in server.csr \
+            -CA intermediate-cert.pem \
+            -CAkey intermediate-key.pem \
+            -CAcreateserial -out server-cert.pem \
+            -copy_extensions=copyall
 
     echo "generating client certificate and signing it"
-    openssl x509 -days 65536 -req -in client.csr -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out client-cert.pem -extensions req_ext -extfile ../config/client.cnf
-
-    echo "verifying generated certificates"
-    openssl verify -CAfile ca-cert.pem server-cert.pem
+    openssl x509 -days 65536 \
+            -req -in client.csr \
+            -CA ca-cert.pem \
+            -CAkey ca-key.pem \
+            -CAcreateserial -out client-cert.pem \
+            -copy_extensions=copyall
+
+    touch server-chain.pem
+    cat server-cert.pem >> server-chain.pem
+    cat intermediate-cert.pem >> server-chain.pem
+    cat ca-cert.pem >> server-chain.pem
+
+    echo "verifying server certificates"
+    openssl verify -CAfile ca-cert.pem intermediate-cert.pem
+    openssl verify -CAfile ca-cert.pem -untrusted intermediate-cert.pem server-cert.pem
+
+    echo "verifying client certificates"
     openssl verify -CAfile ca-cert.pem client-cert.pem
 
     echo "cleaning up temporary files"
     rm server.csr
+    rm intermediate.csr
     rm client.csr
     rm ca-key.pem
 
     cd ..
 }
 
-if [[ $1 != "clean" ]] 
+if [[ $1 != "clean" ]]
 then
+    cert-gen ec 256 ecdsa256
     cert-gen ec 384 ecdsa384
     cert-gen rsa 2048 rsa2048
     cert-gen rsa 3072 rsa3072
     cert-gen rsa 4096 rsa4096
-else 
+else
     echo "cleaning certs"
     rm -rf ecdsa*/ rsa*/
 fi

bindings/rust/bench/src/harness.rs

@@ -25,7 +25,7 @@ impl PemType {
     fn get_filename(&self) -> &str {
         match self {
             PemType::ServerKey => "server-key.pem",
-            PemType::ServerCertChain => "server-cert.pem",
+            PemType::ServerCertChain => "server-chain.pem",
             PemType::ClientKey => "client-key.pem",
             PemType::ClientCertChain => "client-cert.pem",
             PemType::CACert => "ca-cert.pem",
@@ -40,6 +40,7 @@ pub enum SigType {
     Rsa4096,
     #[default]
     Ecdsa384,
+    Ecdsa256,
 }
 
 impl SigType {
@@ -49,6 +50,7 @@ impl SigType {
             SigType::Rsa3072 => "rsa3072",
             SigType::Rsa4096 => "rsa4096",
             SigType::Ecdsa384 => "ecdsa384",
+            SigType::Ecdsa256 => "ecdsa256",
         }
     }
 }