Server certificate hostname validation implementation not working when SAN without DNS, IP, URI is present.

[mman]

Summary

I am using custom certificates with CN=hostname and one additional properly tagged SANs (rfc822Name). The new hostname verification configured via mbedlts_ssl_set_hostname fails for such certificates.

System information

Mbed TLS version (number or commit id): 3.6.3

Expected behavior

I am not an expert on x509, but I assume a certificate with properly formatted subject with CN=hostname and SAN without additional DNS name or IP address should validate.

Actual behavior

Validation of server certificate fails when it contains properly defined CN=hostname in subject, and contains additional SANs, none of which are IP, DNS, or URI.

Additional information

The actual validation of cert and hostname happens here:

mbedtls/library/ssl_tls.c

Line 8823 in 6bf29fd

ret = mbedtls_x509_crt_verify_restartable(

When presence of SANs is detected, the check ignores subject CN entirely, and examines the SANs only.

mbedtls/library/x509_crt.c

Lines 2960 to 2972 in 6bf29fd

	if (crt->ext_types & MBEDTLS_X509_EXT_SUBJECT_ALT_NAME) {
	if (x509_crt_check_san(&crt->subject_alt_names, cn, cn_len) == 0) {
	return;
	}
	} else {
	for (name = &crt->subject; name != NULL; name = name->next) {
	if (MBEDTLS_OID_CMP(MBEDTLS_OID_AT_CN, &name->oid) == 0 &&
	x509_crt_check_cn(&name->val, cn, cn_len) == 0) {
	return;
	}
	}
	}

And when SANs are analyzed, only the IP, DNS, and URI tags are examined. Presence of none is indicated as an error and certificate is flagged as untrusted.

mbedtls/library/x509_crt.c

Lines 2920 to 2930 in 6bf29fd

	case MBEDTLS_X509_SAN_DNS_NAME:
	if (x509_crt_check_cn(&cur->buf, cn, cn_len) == 0) {
	return 0;
	}
	break;
	case MBEDTLS_X509_SAN_IP_ADDRESS:
	san_ip = 1;
	break;
	case MBEDTLS_X509_SAN_UNIFORM_RESOURCE_IDENTIFIER:
	san_uri = 1;
	break;

Suggested fix

Unless I am missing something from the X509 standard, which I may of course, I believe that subject CN should be validated in addition to any additional DNS, IP, or URI SANs... and result presented when any of these pass.

[mman]

I believe the suggested fix may look like this:

First we walk the list of SANs, and try to find a match among DNS, IP, URI, if valid match found -> exit.

Second, try to validate against subject, if match found -> exit.

Otherwise fail.

--- x509_crt.c.orig	2025-03-26 22:06:40
+++ x509_crt.c	2025-03-26 22:10:02
@@ -2988,18 +2988,18 @@
     const mbedtls_x509_name *name;
     size_t cn_len = strlen(cn);
 
+    // iterate over SANs and try to validate cn against DNS, IP, URI
     if (crt->ext_types & MBEDTLS_X509_EXT_SUBJECT_ALT_NAME) {
         if (x509_crt_check_san(&crt->subject_alt_names, cn, cn_len) == 0) {
             return;
         }
-    } else {
-        for (name = &crt->subject; name != NULL; name = name->next) {
-            if (MBEDTLS_OID_CMP(MBEDTLS_OID_AT_CN, &name->oid) == 0 &&
-                x509_crt_check_cn(&name->val, cn, cn_len) == 0) {
-                return;
-            }
+    }
+    // validate cn against certificate subject 
+    for (name = &crt->subject; name != NULL; name = name->next) {
+        if (MBEDTLS_OID_CMP(MBEDTLS_OID_AT_CN, &name->oid) == 0 &&
+            x509_crt_check_cn(&name->val, cn, cn_len) == 0) {
+            return;
         }
-
     }
 
     *flags |= MBEDTLS_X509_BADCERT_CN_MISMATCH;

[davidhorstmann-arm]

Looking at X.509, it looks like there's a small amount of subtlety here. From X.509 9.3.2.1 (Subject alternative name extension):

This extension may, at the option of the issuing CA, be either flagged as critical or as non-critical. A relying party that supports this extension is not required to be able to process all name forms. If the extension is flagged as critical, at least one of the name forms that is present shall be recognized and processed, otherwise the public-key certificate shall be considered invalid. Apart from the preceding restriction, a relying party is permitted to ignore any name with an unrecognized or unsupported name form.

It looks like if the subjectAltName is flagged as critical, we should fail if we do not find a match there. Otherwise we can check the subject name.

Also in 7.2.1:

If the public-key certificate is an end-entity public-key certificate (see clause 7.4), then the distinguished name may be an empty sequence providing that the subjectAltName extension is present and is flagged as critical (see clause 9.3.2.1).

Which implies that we should always check the subject name ('distinguished name') for a match if the subjectAltName is marked as non-critical and we don't find a match there.

So what you're proposing looks correct to me, except that we shouldn't look at the subject name if the subjectAltName is marked as critical.

[davidhorstmann-arm]

This looks relatively straightforward, but could create some challenges with testing.

Steps to complete:

• Add tests verifying certificates (or at least calling x509_crt_verify_name()) for the following cases:
  • Cert with a subjectAltName extension that doesn't match, not flagged as critical, and a CN that does match (should pass verification)
  • Cert with a subjectAltName extension that doesn't match, flagged as critical, and a CN that does match (should fail verification)
  • End-entity cert (CA=false) with a subjectAltName extension that does match, not flagged as critical, and an empty subject name (should fail)
  • CA cert (CA=true) with a subjectAltName extension that does match, flagged as critical, and an empty subject name (should fail)
  • End-entity cert with a subjectAltName extension that does match, flagged as critical, and an empty subject name (should pass)
• Rewrite the name-checking logic to make those tests pass

I'm going to label it size-s. I'd say on the larger end of the scale given the testing that needs to be added.