IsCa::ExplicitNoCA flag produces invalid DER

[GermanCoding]

Description of the bug

When rcgen generates a certificate with CertificateParams.is_ca = IsCa::ExplicitNoCa, rcgen produces a DER encoding that is invalid. The root cause is that it writes an explicit false to the sequence, which is not allowed per X.690 §11.5:

11.5 Set and sequence components with default value
The encoding of a set value or sequence value shall not include an encoding for any component value which is equal to its
default value.

Strict parsers which require DER conformance will reject such a certificate. An example is python's cryptography, which will not load rcgen's certificate.

Expected behaviour

ExplicitNoCA should write an empty sequence, omitting the false boolean. This can also be checked by cross-verifying with any public leaf certificate (such as one issued by Let's Encrypt or another public CA). The false boolean should never be written explicitly.

LLM Disclosure

This bug was found by an automated code review made by Claude Opus 5, verified by a human (me).

Reproduction

I've attached a full reproduction example. This was written entirely by Claude Sonnet 5, so it's much more verbose than it needs to be, but I believe it shows the error quite clearly.

Cargo.toml

[package]
name = "rcgen-repro"
version = "0.1.0"
edition = "2021"

[dependencies]
rcgen = { version = "0.14.8", default-features = false, features = ["aws_lc_rs", "pem", "x509-parser"] }

src/main.rs

//! Reproducer: `IsCa::ExplicitNoCa` writes a non-DER `basicConstraints` extension.
//!
//! X.690 section 11.5 ("Set and sequence components with default value"):
//!
//!   "the encoding of a set value or sequence value shall not include an encoding for any
//!    component value which is equal to its default value."
//!
//! `BasicConstraints` (RFC 5280 section 4.2.1.9) is:
//!
//!   BasicConstraints ::= SEQUENCE {
//!        cA                      BOOLEAN DEFAULT FALSE,
//!        pathLenConstraint       INTEGER (0..MAX) OPTIONAL }
//!
//! so a certificate with `cA = FALSE` and no `pathLenConstraint` must, in DER, encode
//! `basicConstraints` as an *empty* SEQUENCE (`30 00`). `IsCa::ExplicitNoCa` instead writes the
//! `cA` boolean explicitly (`30 03 01 01 00`), which is valid BER but not valid DER - and X.509
//! certificates are DER, not merely BER.
//!
//! This program builds a certificate with `IsCa::ExplicitNoCa`, prints the raw bytes of its
//! `basicConstraints` extension, and writes the PEM to `leaf.pem` (alongside the issuing CA in
//! `ca.pem`) for `check_with_python_cryptography.py` to load.

use rcgen::{BasicConstraints, CertificateParams, DistinguishedName, DnType, IsCa, KeyPair, KeyUsagePurpose};

fn main() {
    // A minimal CA to sign the leaf with - `Issuer::from_ca_cert_pem` needs a real issuer.
    let ca_key = KeyPair::generate().unwrap();
    let mut ca_params = CertificateParams::default();
    ca_params.is_ca = IsCa::Ca(BasicConstraints::Unconstrained);
    ca_params.key_usages = vec![KeyUsagePurpose::KeyCertSign, KeyUsagePurpose::CrlSign];
    ca_params.distinguished_name = {
        let mut name = DistinguishedName::new();
        name.push(DnType::CommonName, "repro CA");
        name
    };
    let ca_cert = ca_params.self_signed(&ca_key).unwrap();

    // The leaf: an ordinary end-entity certificate, `IsCa::ExplicitNoCa` because that is the
    // obvious choice for "this is not a CA, and say so explicitly" - it is not a niche or
    // deprecated corner of the API. https://docs.rs/rcgen describes it as "The certificate can
    // only sign itself, adding the extension and CA:FALSE" with nothing marking it non-DER.
    let leaf_key = KeyPair::generate().unwrap();
    let mut leaf_params = CertificateParams::default();
    leaf_params.is_ca = IsCa::ExplicitNoCa;
    leaf_params.distinguished_name = {
        let mut name = DistinguishedName::new();
        name.push(DnType::CommonName, "repro leaf");
        name
    };

    let issuer = rcgen::Issuer::from_ca_cert_pem(&ca_cert.pem(), ca_key).unwrap();
    let leaf_cert = leaf_params.signed_by(&leaf_key, &issuer).unwrap();

    let der = leaf_cert.der();
    let (_, parsed) = x509_parser_bytes(der);
    println!("basicConstraints content octets: {parsed:02x?}");
    println!(
        "  -> a DER-conformant encoder must write `30 00` (empty SEQUENCE) here, not `{parsed:02x?}`"
    );

    std::fs::write("ca.pem", ca_cert.pem()).unwrap();
    std::fs::write("leaf.pem", leaf_cert.pem()).unwrap();
    println!("\nWrote ca.pem and leaf.pem - now run: python check_with_python_cryptography.py");
}

/// Finds the `basicConstraints` extension (OID 2.5.29.19) inside the DER and returns its content
/// octets, by hand - so this reproducer has no dependency on any ASN.1/X.509 parsing crate and
/// cannot be accused of a parsing bug of its own. `basicConstraints` is `30 03 06 03 55 1d 13`
/// (SEQUENCE { OID 2.5.29.19 ... }) as an extension entry; the payload is the OCTET STRING right
/// after the OID, itself containing the BasicConstraints SEQUENCE.
fn x509_parser_bytes(der: &[u8]) -> ((), Vec<u8>) {
    const BASIC_CONSTRAINTS_OID: &[u8] = &[0x06, 0x03, 0x55, 0x1d, 0x13];
    let position = der
        .windows(BASIC_CONSTRAINTS_OID.len())
        .position(|window| window == BASIC_CONSTRAINTS_OID)
        .expect("basicConstraints OID must be present");
    let mut cursor = position + BASIC_CONSTRAINTS_OID.len();
    // Optional BOOLEAN critical (01 01 ff) right after the OID.
    if der[cursor] == 0x01 {
        cursor += 3;
    }
    // OCTET STRING wrapping the extnValue.
    assert_eq!(der[cursor], 0x04, "expected an OCTET STRING tag");
    cursor += 1;
    let octet_len = der[cursor] as usize;
    cursor += 1;
    let octet_string = &der[cursor..cursor + octet_len];
    // The BasicConstraints SEQUENCE itself: tag 0x30, one length byte (short form - always true
    // here, content is at most a few bytes), then the content octets.
    assert_eq!(octet_string[0], 0x30, "expected a SEQUENCE tag");
    let seq_len = octet_string[1] as usize;
    ((), octet_string[2..2 + seq_len].to_vec())
}

Run with with cargo r. It will produce ca.pem and leaf.pem. Example output:

basicConstraints content octets: [01, 01, 00]
  -> a DER-conformant encoder must write `30 00` (empty SEQUENCE) here, not `[01, 01, 00]`

Wrote ca.pem and leaf.pem - now run: python check_with_python_cryptography.py

Now try to load the certificate with python's cryptography:

check_with_python_cryptography.py

"""Loads leaf.pem (written by `cargo run`) with the `cryptography` package's X.509 parser.

`cryptography` rejects the whole certificate because `basicConstraints` explicitly encodes
`cA = FALSE`, which X.690 section 11.5 forbids for a DEFAULT-valued component in DER.

    pip install cryptography
    python check_with_python_cryptography.py
"""

from cryptography import x509

with open("leaf.pem", "rb") as f:
    pem = f.read()

print("Loading leaf.pem with cryptography's X.509 parser...")
# load_pem_x509_certificate itself only parses the outer structure; the strict ASN.1 extension
# decode - and therefore the DER violation - happens lazily, the first time an extension is
# actually read.
cert = x509.load_pem_x509_certificate(pem)
try:
    constraints = cert.extensions.get_extension_for_class(x509.BasicConstraints)
    print("Loaded fine - unexpected, the bug may already be fixed:")
    print(constraints)
except Exception as err:
    print(f"REJECTED: {type(err).__name__}: {err}")

Produces:

Loading leaf.pem with cryptography's X.509 parser...
REJECTED: ValueError: error parsing asn1 value: ParseError { kind: EncodedDefault, location: ["BasicConstraints::ca"] }