Unsupported Certificate due to unknown_critical_extension in OTP 26.2.5 with RabbitMQ

[nvinzens]

Describe the bug
After upgrading OTP from 25.3.1 to 26.2.5 we can no longer establish TLS sessions in rabbitMQ. Current assumption is that the cause is that we are using a ca cert that contains a critical extension.
Related logs:

<29>May 23 10:58:48 REDACTED rabbitmq-server[19980] <0.33281.0> - TLS server: In state wait_cert at ssl_handshake.erl:2117 generated SERVER ALERT: Fatal - Unsupported Certificate
<29>May 23 10:58:48 REDACTED rabbitmq-server[19980] <0.33287.0> - TLS server: In state wait_cert at ssl_handshake.erl:2117 generated SERVER ALERT: Fatal - Unsupported Certificate

We were using the same rabbitMQ version with both versions. Downgrading the Erlang version again fixes the issue.

To Reproduce
Use a ca cert that looks like below (redacted some parts), the relevant part should be X509v3 Basic Constraints:

# openssl x509 -inform pem -in /opt/OSAGrabbitmq/etc/ca_server_cert.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            REDACTED
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: emailAddress = REDACTED, CN = REDACTED, OU = REDACTED, O = REDACTED, L = REDACTED, ST = ., C = REDACTED
        Validity
            Not Before: Apr  3 09:59:05 2020 GMT
            Not After : May 21 09:59:05 2030 GMT
        Subject: emailAddress = REDACTED, CN = REDACTED, OU = REDACTED, O = REDACTED, L = Zurich, ST = ., C = REDACTED
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (4096 bit)
                Modulus:
                    REDACTED
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                REDACTED
            X509v3 Authority Key Identifier:
                keyid:REDACTED
            X509v3 Basic Constraints: critical
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
 REDACTED
#

Expected behavior
The TLS connection should work with the same critical extension in versions 26.2.5 and 25.3.1.

Affected versions
26.2.5

Additional context
I'm very unfamiliar with Erlang in general but the relevant code seems to be here on master: https://github.com/erlang/otp/blob/master/lib/ssl/src/ssl_handshake.erl#L2168

The relevant part of the RabbitMQ config that configures TLS:

# ssl
ssl_cert_login_from = common_name
ssl_options.cacertfile = /opt/OSAGrabbitmq/etc/ca_client_cert.pem
ssl_options.certfile = /opt/OSAGrabbitmq/etc/server_cert.pem
ssl_options.keyfile = /opt/OSAGrabbitmq/etc/server_key.pem
ssl_options.verify = verify_peer
ssl_options.fail_if_no_peer_cert = true

If I can provide any additional needed information please let me know.

[IngelaAndin]

@nvinzens I think I misread it the first time so I am updating my comment. It is the not the server chain that is the problem it is the client chain. The code that determines if there is a un-handled critical extension in a certificate chain is located in the public_key application.

[IngelaAndin]

You could try tracing:

dbg:tracer(). dbg:p(all, call). dbg:tpl(pubkey_cert, verify_fun).

[IngelaAndin]

@nvinzens Do you have any addition information? We need to know what the client certificate looks like to determine why it is considered unsupported. I can also recommend OTP-26.2.5.1 to probably get some more information from the logs.