WIP: Fix keyCertSign checks

[levitte]

Trying to check CA usage in x509v3_cache_extensions wasn't a good idea.

However, the keyCertSign check in check_ca wasn't quite right either

RFC 5280 says this in 4.2.1.3 "Key Usage":

The keyCertSign bit is asserted when the subject public key is
used for verifying signatures on public key certificates. If the
keyCertSign bit is asserted, then the cA bit in the basic
constraints extension (Section 4.2.1.9) MUST also be asserted.

check_ca did check these relevant combinations:

1. keyCertSign bit set in keyUsage, no check of basicConstraints.
2. cA bit set in basicConstraints, no check of keyUsage.
3. basicConstraints not present => keyCertSign bit set in keyUsage.

Now we change 1 to:

1. keyCertSign bit set in keyUsage => cA bit set in basicConstraints.

Fixes #1418

[levitte]

Should this go in 1.1.0 as well?

[levitte]

HELP NEEDED

There is a test_verify failure in Travis (and I suppose Appveyor as well):

CN = IETF Test Demo
error 18 at 0 depth lookup: self signed certificate
error ../../../_srcdist/test/certs/ee-ed25519.pem: verification failed
../../util/shlib_wrap.sh ../../apps/openssl verify -auth_level 1 -purpose sslserver -trusted ../../../_srcdist/test/certs/root-ed25519.pem ../../../_srcdist/test/certs/ee-ed25519.pem => 2
not ok 134 - ED25519 signature

Looking more closely at this, test/certs/ee-ed25519.pem does look... interesting:

: ; openssl x509 -text -noout -in test/certs/ee-ed25519.pem 
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 6197312946105598768 (0x5601474a2a8dc330)
        Signature Algorithm: ED25519
        Issuer: CN = IETF Test Demo
        Validity
            Not Before: Aug  1 12:19:24 2016 GMT
            Not After : Dec 31 23:59:59 2040 GMT
        Subject: CN = IETF Test Demo
        Subject Public Key Info:
            Public Key Algorithm: X25519
                X25519 Public-Key:
                pub:
                    85:20:f0:09:89:30:a7:54:74:8b:7d:dc:b4:3e:f7:
                    5a:0d:bf:3a:0d:26:38:1a:f4:eb:a4:a9:8e:aa:9b:
                    4e:6a
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Key Usage: 
                Key Agreement
            X509v3 Subject Key Identifier: 
                9B:1F:5E:ED:ED:04:33:85:E4:F7:BC:62:3C:59:75:B9:0B:C8:BB:3B
    Signature Algorithm: ED25519
         af:23:01:fe:dd:c9:e6:ff:c1:cc:a7:3d:74:d6:48:a4:39:80:
         82:cd:db:69:b1:4e:4d:06:ec:f8:1a:25:ce:50:d4:c2:c3:eb:
         74:6c:4e:dd:83:46:85:6e:c8:6f:3d:ce:1a:18:65:c5:7a:c2:
         7b:50:a0:c3:50:07:f5:e7:d9:07

This certainly looks like a self-issued certificate, doesn't it? There's no AKID saying anything else, so the assumption is that it is self-signed as well.

[levitte]

The problem I had here gets resolved with #7922

[DDvO]

The above cert is definitely self-issued since its subject and issuer are identical. Moreover, since consequently its SKID und AKID would be equal, the latter is left left out (almost in accordance with RFC 5280 section 4.2.1.1, where I'd say the exception there should refer to self-issued rather than self-signed certs).

Yet it turns out that the cert is not self-signed (or at least OpenSSL fails to verify its signature, which needs to be triggered explicitly):

apps/openssl verify -check_ss_sig -trusted test/certs/ee-ed25519.pem test/certs/ee-ed25519.pem

yields

error 7 at 0 depth lookup: certificate signature failure

For more information about this cert, see https://tools.ietf.org/html/rfc8410#section-10.2.

[DDvO]

I've just checked: the signature error is thrown at this point in a_verify.c:

        if (!pkey->ameth || !pkey->ameth->item_verify) {
            ASN1err(ASN1_F_ASN1_ITEM_VERIFY,
                    ASN1_R_UNKNOWN_SIGNATURE_ALGORITHM);

because pkey->ameth->item_verify is NULL.
Having a closer look at the algorithm type of the public key of this cert, which is X25519, reveals that this algorithm is not for signatures but for key agreement algorithm, and this is why the self-signature verification must fail.

To conclude, the above test cert is self-issued but not self-signed.
Which makes the test case

    ok(verify("ee-ed25519", "sslserver", ["root-ed25519"], []),
       "ED25519 signature");

even more strange than commented so far in #1418 (comment) .

[levitte]

Actually, if you look at https://tools.ietf.org/html/rfc8410#section-10.2, you'll note that there's a public key and a private key there as well. I assume that those were used to sign the certificate.
The public key there is the same that's embedded in root-ed25519.pem, which is the reason why it's used as a trust anchor.

Now, ee-ed25519.pem (which is taken directly from that RFC) is constructed a bit weirdly. For all intents and purposes, it's self issued for sure, and as far as RFC 5280 goes, there should be an AKID in it indicating that it's signed by another key.

So bottom line, there isn't much information telling us that ee-ed2219.pem isn't a self-signed certificate, save for doing a verification against the included public key... I'm not sure we should do that...

[levitte]

You know what, I do believe that ee-ed25519.pem is incorrect. From https://tools.ietf.org/html/rfc8410#section-10.2:

   An example of an Ed25519 public key:

         Public Key Information:
             Public Key Algorithm: Ed25519
             Algorithm Security Level: High

         Public Key Usage:

         Public Key ID: 9b1f5eeded043385e4f7bc623c5975b90bc8bb3b

         -----BEGIN PUBLIC KEY-----
         MCowBQYDK2VwAyEAGb9ECWmEzf6FQbrBZ9w7lshQhqowtrbLDFw4rXAxZuE=
         -----END PUBLIC KEY-----

Now, the certificate:

   An example of a self-issued PKIX certificate using Ed25519 to sign an
   X25519 public key would be:
...
   196  32:         SEQUENCE {
   198   3:           OBJECT IDENTIFIER subjectKeyIdentifier (2 5 29 14)
   203   1:           BOOLEAN FALSE
   206  22:           OCTET STRING, encapsulates {
   208  20:             OCTET STRING
          :               9B 1F 5E ED ED 04 33 85 E4 F7 BC 62 3C 59 75
          :               B9 0B C8 BB 3B
          :             }
          :           }
...

Notice anything weird with that SKID?

I've always wondered why it was there, for what purpose... but y'know what? I think "they" screwed up, the SKID should really be an AKID with that key id. That would make a lot more sense.

[levitte]

(also, it must be late at night, but I seem to get a different key ID for that public key than they do... so if someone else could verify that calculation, that would be great)

[richsalz]

you could write to the author and start a discussion. it might be that an errata will have to be filed :)

[bernd-edlinger]

@levitte your self-signed certificate looks invalid,
the algorthm is X25519 but it should be ED25519, like this one:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            12:7b:a9:24:ae:ca:33:4d:9b:16:ff:fa:64:f0:a7:f2
        Signature Algorithm: ED25519
        Issuer: CN = "Softing UA C++ SDK TestServer", O = Softing
        Validity
            Not Before: Dec 21 11:53:37 2018 GMT
            Not After : Jan  4 11:53:37 2019 GMT
        Subject: CN = "Softing UA C++ SDK TestServer", O = Softing
        Subject Public Key Info:
            Public Key Algorithm: ED25519
                ED25519 Public-Key:
                pub:
                    6a:63:4f:ec:70:27:b5:97:f8:99:96:6e:cc:17:a0:
                    38:fa:63:48:92:3a:e4:43:92:44:5b:10:a0:e1:9a:
                    9f:25
        X509v3 extensions:
            X509v3 Subject Alternative Name: 
                URI:urn:w-ed1/Softing/OpcUa/TestServer, IP Address:127.0.0.1, DNS:localhost
            X509v3 Basic Constraints: 
                CA:FALSE
            X509v3 Subject Key Identifier: 
                63:51:63:4A:3A:A0:5D:E4:3F:68:7B:DD:6E:30:CE:5D:36:24:09:B6
            X509v3 Authority Key Identifier: 
                keyid:63:51:63:4A:3A:A0:5D:E4:3F:68:7B:DD:6E:30:CE:5D:36:24:09:B6

            X509v3 Key Usage: 
                Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment, Certificate Sign, CRL Sign
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
    Signature Algorithm: ED25519
         e8:17:5f:26:3a:81:12:53:c0:ea:8c:07:ab:4e:b7:56:76:83:
         9d:22:03:11:53:36:0c:97:4d:aa:3c:d1:1f:97:d1:8c:ce:c3:
         b0:db:18:77:c7:c7:67:6c:38:2b:4c:8f:c6:e1:c6:e3:2c:85:
         a4:83:f7:e1:33:22:fc:cc:d9:0d

[DDvO]

@bernd-edlinger, the algorithm X25519 given for the key type is fine because the cert is meant to be for an X25519 key agreement key - see https://tools.ietf.org/html/rfc8410#section-10.2 .

@levitte, I confirm your analysis (and need to withdraw one sentence I wrote above concerning the AKID):
the ID of the key used for signing should have been given in the cert as the AKID, and the SKID should be something different, namely, the hash of the given public key. In this way it would also become clear from viewing the cert that it is self-issued but not self-signed.

[DDvO]

I tried to verify the cert signature with the mentioned public key:

echo "-----BEGIN PUBLIC KEY-----
MCowBQYDK2VwAyEAGb9ECWmEzf6FQbrBZ9w7lshQhqowtrbLDFw4rXAxZuE=
-----END PUBLIC KEY-----" > test/certs/root-ed25519-pub.pem
apps/openssl pkeyutl -verify -in test/certs/ee-ed25519.pem -pubin -inkey test/certs/root-ed25519-pub.pem

but got:

pkeyutl: Error initializing context
139732907656960:error:0608F096:digital envelope routines:EVP_PKEY_verify_init:operation not supported for this keytype:crypto/evp/pmeth_fn.c:74:

At least, the encoding of the public key is correct, as confirmed by:

apps/openssl pkey -pubin -in test/certs/root-ed25519-pub.pem  -noout -text

which yields:

ED25519 Public-Key:
pub:
    19:bf:44:09:69:84:cd:fe:85:41:ba:c1:67:dc:3b:
    96:c8:50:86:aa:30:b6:b6:cb:0c:5c:38:ad:70:31:
    66:e1

Does anyone know/have a signature verifier that can handle ED25519 to verify that the cert is signed using the private key corresponding to test/certs/root-ed25519.pem?

[DDvO]

I get the same result when trying the private key as input for the verification:

echo "-----BEGIN PRIVATE KEY-----
> MC4CAQAwBQYDK2VwBCIEINTuctv5E1hK1bbY8fdp+K06/nwoy/HU++CXqI9EdVhC
> -----END PRIVATE KEY-----" >test/certs/root-ed25519-priv.pem
apps/openssl pkeyutl -verify -in test/certs/ee-ed25519.pem -inkey test/certs/root-ed25519-priv.pem

yields

pkeyutl: Error initializing context
140091448231680:error:0608F096:digital envelope routines:EVP_PKEY_verify_init:operation not supported for this keytype:crypto/evp/pmeth_fn.c:74:

[DDvO]

(also, it must be late at night, but I seem to get a different key ID for that public key than they do... so if someone else could verify that calculation, that would be great)

For SIKD and AKID there is not really right or wrong. As written in RFC 5280 sections 4.2.1.1. and 4.2.1.2, they can be basically anything unique, while the recommendation is to use the SHA-1 hash of the value of the BIT STRING representation of the public key (excluding the tag, length, and number of unused bits).

[bernd-edlinger]

the only command I can think of that is able to check the ED25519 signature of an X509 is:
openssl verify -trust self_signed.pem -check_ss_sig self_signed.pem

[bernd-edlinger]

I still fail to see how the signature can be verified, the certificate has subject=issuer,
so that implies that the signature was created by the same certificate,
but it does not have
X509v3 Key Usage: critical
Digital Signature
and the key is unable to sign anything because it is X25519.

Everything would instantly make sense, if the key agreement was signed by a CA.
What is a key agreement worth when there is no signature that one can verify?

[DDvO]

I still fail to see how the signature can be verified, the certificate has subject=issuer,
so that implies that the signature was created by the same certificate,

One needs to be careful here: self-issued does not necessarily imply self-signed! I myself learned this neat distinction just this week.

See RFC 5280 section 3.2:

Self-issued certificates are CA certificates in which
the issuer and subject are the same entity. Self-issued certificates
are generated to support changes in policy or operations. Self-
signed certificates are self-issued certificates where the digital
signature may be verified by the public key bound into the
certificate.

The two different terms are used very deliberately throughout this RFC.

In our case here the EE cert happens to have the same subject as the issuer (so is self-issued), and its is signed by the same issuer as the root cert, but only the root cert is self-signed.

Note that the same cert signer (same identity) may have multiple key pairs, for potentially different purposes. This is why the presence of the Authority Key Identifier (AKID) is important here to differentiate the signing keys. The cert, as given in https://tools.ietf.org/html/rfc8410#section-10.2 fails to include it.

This is the actual reason why the test case

    # ED25519 certificate from draft-ietf-curdle-pkix-04
    ok(verify("ee-ed25519", "sslserver", ["root-ed25519"], []),
       "ED25519 signature");

meanwhile (correctly) fails: it should not fail because the cert does not allow for key usage keyCertSign but because the chain to the trusted root cannot be established due to the missing AKID, which misleads the verification procedure in assuming that the cert is self-signed.

[DDvO]

What is a key agreement worth when there is no signature that one can verify?

For the given cert, the intended key usage is definitely just key agreement. The private key belonging to the certified public key has not been used to sign this cert (and does not need to be used for signing). This cert happens to be signed by the same issuer, but using a different key: presumably the key certified in the CA cert test/certs/root-ed25519.pem).

[DDvO]

BTW, it looks to me like OpenSSL so far simply does not (fully) support verifying ED25519 signatures.
See also letsencrypt/boulder#3649 and #487.

[bernd-edlinger]

BTW, it looks to me like OpenSSL so far simply does not (fully) support verifying ED25519 signatures.

I am pretty sure 1.1.1 does support verifying PureEDDSA, because I have implemented
this recently, it is simple as this:

pEcxSignature = EVP_MD_CTX_new();
EVP_DigestSignInit(pEcxSignature, NULL, NULL, NULL, pEcxPrivateKey);
EVP_DigestSign(pEcxSignature, pSignature, &siglen, pData, datalen);

pEcxSignature = EVP_MD_CTX_new();
EVP_DigestVerifyInit(pEcxSignature, NULL, NULL, NULL, pEcxPublicKey);
EVP_DigestVerify(pEcxSignature, pSignature, siglen, pData, datalen);

But maybe there is no command which verifies EDDSA

[bernd-edlinger]

The ee certificate can't be self-signed because the public key algorihm X25519 does
not support signatures. Also I wonder why it is no longer the case that a certificate
needs the key usage bit for Certificate Sign to sign its own certificate.

[bernd-edlinger]

Now we change 1 to:
keyCertSign bit set in keyUsage => cA bit set in basicConstraints.

So, as a matter of fact we have now a lot of self signed certificates, that are not being used as CA
which have the keyCertSig bit set and the basicConstraints.CA=false.
Does this mean those certificate will now be invalid?

[bernd-edlinger]

how about this:

--- a/crypto/x509v3/v3_purp.c
+++ b/crypto/x509v3/v3_purp.c
@@ -463,8 +463,16 @@ static void x509v3_cache_extensions(X509 *x)
     if (!X509_NAME_cmp(X509_get_subject_name(x), X509_get_issuer_name(x))) {
         x->ex_flags |= EXFLAG_SI;
         /* If SKID matches AKID also indicate self signed */
-        if (X509_check_akid(x, x->akid) == X509_V_OK)
-            x->ex_flags |= EXFLAG_SS;
+        if (X509_check_akid(x, x->akid) == X509_V_OK) {
+            X509_PUBKEY *xpkey = X509_get_X509_PUBKEY(x);
+            if (xpkey != NULL) {
+                EVP_PKEY *pkey = X509_PUBKEY_get0(xpkey);
+                if (pkey != NULL
+                        && EVP_PKEY_id(pkey) != EVP_PKEY_X25519
+                        && EVP_PKEY_id(pkey) != EVP_PKEY_X448)
+                    x->ex_flags |= EXFLAG_SS;
+            }
+        }
     }
     x->altname = X509_get_ext_d2i(x, NID_subject_alt_name, NULL, NULL);
     x->nc = X509_get_ext_d2i(x, NID_name_constraints, &i, NULL);

It can't be self-signed when we are unable to verify the signature using the public key.

[DDvO]

Adding references to specific crypto algorithms would add quite some clutter to the verification functions, which are already very hard to follow. If done for these two algorithms, then consequently putting in many more such exceptions etc. would be in order.

The RFC 5280 does not do this, and I'd recommend not to do this in the OpenSSL implementation, either.

[DDvO]

I am pretty sure 1.1.1 does support verifying PureEDDSA, because I have implemented
this recently, it is simple as this:

Nice.

But maybe there is no command which verifies EDDSA

I've just tried with the latest 3.0.0-dev, for instance like this:

apps/openssl verify -check_ss_sig -trusted test/certs/root-ed25519.pem test/certs/root-ed25519.pem

and it works :)

@levitte, with this very fresh OpenSSL version one can confirm that (the private key related to) the root cert has been used to sign the weird ee cert:

apps/openssl verify -check_ss_sig -trusted test/certs/root-ed25519.pem test/certs/root-ed25519.pem

[DDvO]

The ee certificate can't be self-signed because the public key algorihm X25519 does not support signatures.

Yes.

Also I wonder why it is no longer the case that a certificate needs the key usage bit for Certificate Sign to sign its own certificate.

RFC 5280 is inconsistent in this respect: it requires keyCertSign for any(!) key used for signing certs, but on the other hand for self-signed EE certs the presence of keyCertSign is not checked according to the algorithm provided in https://tools.ietf.org/html/rfc5280#section-6.1. The most reasonable resolution is to follow the stated algorithm (and not require keyCertSign for such certs).
We have discussed this very recently in #1418.

Now we change 1 to:
keyCertSign bit set in keyUsage => cA bit set in basicConstraints.

So, as a matter of fact we have now a lot of self signed certificates, that are not being used as CA which have the keyCertSig bit set and the basicConstraints.CA=false.

For good (security) reason, such end-entity certs should indeed have CA=false.

Does this mean those certificate will now be invalid?

Yes, when they allow keyCertSign this is (as mentioned also in #1418) not conforming with RFC 5280, which (correctly) demands CA=true for all certs having the keyCertSign set.

[bernd-edlinger]

So will this be a breaking change, since the current state of affairs forced us to set keyCertSign in the
first place.

[DDvO]

So will this be a breaking change, since the current state of affairs forced us to set keyCertSign in the
first place.

When the problem handled in issue #7918 came up for one of my customers some two weeks ago, after finding out what the real reason was behind a rather misleading error: "unable to get local issuer certificate", the solution I suggested spontaneously also was to add keyCertSign to the key usage extension of self-signed end entity certs.

Yet after some discussion with PKI experts we came to the conclusion that its cleaner - and much safer - not to do it. We use a workaround:

X509_VERIFY_PARAM_set_flags(X509_STORE_CTX_get0_param(store_ctx), X509_V_FLAG_PARTIAL_CHAIN)

until #7918 is (fully) fixed and the fix has become available in our developments.

[bernd-edlinger]

Okay, the good news is, I checked the current PR against our software (an OPC/UA SDK)
and it does not immediately break anything, probably because we do not use
X509_V_FLAG_X509_STRICT

However, we had detected this issue years before I joined this project.

Previously all self-signed certificates were correctly using CA=FALSE, and keyCertSign=FALSE,
However now there are now many certificates around with CA=FALSE, and keyCertSign=TRUE,
which is not so easy to undo, because there is an installed basis of zillions of OPC/UA Servers,
and it will take years until all of them are updated to a new openssl version or even
adding a X509_V_FLAG_PARTIAL_CHAIN is near impossible with an embedded device.

[DDvO]

Well, as mentioned certificates with CA=FALSE and keyCertSign set do not comply with RFC 5280.

I should have mentioned that we use the X509_V_FLAG_PARTIAL_CHAIN workaround only if a self-signed EE cert (with no chain, where the same cert should be in the trust store) is to be verified.

[bernd-edlinger]

I am only saying, if a bug like that has been around for four years, you cannot simply
fix it without a bug-to-bug compatibility flag, for those who need it.

[DDvO]

I had a close look again at that strange ED25519 self-issued example cert with X25519 key agreement public key: https://tools.ietf.org/html/draft-ietf-curdle-pkix-04#section-10.2
It turns out that the cert is not RFC 5280 compliant because although being not self-signed it does not have an Authority Key Identifier although required by https://tools.ietf.org/html/rfc5280#section-4.2.1.1

This makes x509v3_cache_extensions() and thus cert_self_signed() believe that the cert is self-signed because a check is missing: a cert cannot be self-signed when the key algorithm of its public key does not support signing.

[DDvO]

The incorrect handling of the weird ED25519 example cert is fixed in #10587.

[t8m]

Are you still interested in having this change included in OpenSSL? If so please rebase the PR to resolve conflicts.

Marking as inactive to be closed at the end of 3.4 dev barring further input.