openssl-3.0.12 cannot decode beijing CA certificate。 #23259

andylee521 · 2024-01-11T01:06:35Z

Hi guys，
I used comand "openssl x509 -in ./bjca.pem -noout -text" to print bjca's certificate content, but the result is that the certificate public key information cannot be decoded。
I analyzed the decoding logic and found that the decoder uses EC and the decoded group is SM2 from this certificate, so an error is returned when judging（function ec_check()）, and the decoding cannot be successful.
Openssl version is 3.0.12, and so do the private key file. I think the root cause is OID:1.2.840.10045.2.1.
How to use the existing logic to be compatible with the old version of the certificate and the existing version of the certificate may be the key to solving this problem.
BJCA.zip

jamuir · 2024-01-11T15:40:23Z

when I try this against master, it works correctly:

$ LD_LIBRARY_PATH=../.. ../../apps/openssl x509 -in BJCA.pem -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 7629940819692799611 (0x69e2fec0170ac67b)
        Signature Algorithm: SM2-with-SM3
        Issuer: C=CN, O=NRCAC, CN=ROOTCA
        Validity
            Not Before: Jul 14 03:11:59 2012 GMT
            Not After : Jul  7 03:11:59 2042 GMT
        Subject: C=CN, O=NRCAC, CN=ROOTCA
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:30:f0:9c:6b:aa:66:81:c7:21:b1:37:f6:52:70:
                    5e:2f:da:ed:a7:89:f0:fa:2b:64:d4:ac:eb:99:b9:
                    ea:a3:4e:65:53:09:30:95:62:be:e0:e2:2b:b4:57:
                    40:aa:74:53:57:b4:3d:bf:58:6d:92:fe:36:4e:c2:
                    2e:b7:37:75:db
                ASN1 OID: SM2
        X509v3 extensions:
            X509v3 Authority Key Identifier: 
                4C:32:B1:97:D9:33:1B:C4:A6:05:C1:C6:E5:8B:62:5B:F0:97:76:58
            X509v3 Basic Constraints: 
                CA:TRUE
            X509v3 Key Usage: 
                Certificate Sign, CRL Sign
            X509v3 Subject Key Identifier: 
                4C:32:B1:97:D9:33:1B:C4:A6:05:C1:C6:E5:8B:62:5B:F0:97:76:58
    Signature Algorithm: SM2-with-SM3
    Signature Value:
        30:45:02:20:1b:56:d2:2d:e3:97:a7:7a:01:f0:7e:db:e7:75:
        be:08:a3:8f:97:63:e4:9e:65:84:ab:f9:4c:86:d9:f6:e4:79:
        02:21:00:da:1c:38:16:c5:61:6d:9c:2a:c1:8c:7d:7a:fd:6d:
        c4:ce:7e:ff:53:f5:63:a3:9c:48:a4:3a:22:56:1b:0b:c2

I will try against 3.0.

jamuir · 2024-01-11T16:03:20Z

I've also tried against 3.0.12, 3.1.4, 3.2.0 and BJCA.pem is successfully parsed and displayed by each openssl version:

openssl-3.0.12$ LD_LIBRARY_PATH=. apps/openssl version 
OpenSSL 3.0.12 24 Oct 2023 (Library: OpenSSL 3.0.12 24 Oct 2023)
openssl-3.0.12$ LD_LIBRARY_PATH=. apps/openssl x509 -in BJCA.pem -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 7629940819692799611 (0x69e2fec0170ac67b)
        Signature Algorithm: SM2-with-SM3
        Issuer: C = CN, O = NRCAC, CN = ROOTCA
        Validity
            Not Before: Jul 14 03:11:59 2012 GMT
            Not After : Jul  7 03:11:59 2042 GMT
...

openssl-3.1.4$ LD_LIBRARY_PATH=. apps/openssl version
OpenSSL 3.1.4 24 Oct 2023 (Library: OpenSSL 3.1.4 24 Oct 2023)
openssl-3.1.4$ LD_LIBRARY_PATH=. apps/openssl x509 -in BJCA.pem -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 7629940819692799611 (0x69e2fec0170ac67b)
        Signature Algorithm: SM2-with-SM3
        Issuer: C = CN, O = NRCAC, CN = ROOTCA
        Validity
            Not Before: Jul 14 03:11:59 2012 GMT
            Not After : Jul  7 03:11:59 2042 GMT
...

openssl-3.2.0$ LD_LIBRARY_PATH=. apps/openssl version
OpenSSL 3.2.0 23 Nov 2023 (Library: OpenSSL 3.2.0 23 Nov 2023)
openssl-3.2.0$ LD_LIBRARY_PATH=. apps/openssl x509 -in BJCA.pem -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 7629940819692799611 (0x69e2fec0170ac67b)
        Signature Algorithm: SM2-with-SM3
        Issuer: C=CN, O=NRCAC, CN=ROOTCA
        Validity
            Not Before: Jul 14 03:11:59 2012 GMT
            Not After : Jul  7 03:11:59 2042 GMT
...

@andylee521 : could you post the output of the error you are seeing?

jamuir · 2024-01-11T16:05:52Z

is it possible that your build of openssl has support for SM2 disabled?

andylee521 · 2024-01-12T00:37:48Z

Hi jamuir，
I'm sorry for trouble you and t8m, I tested it again used 3.0.12, it works.
But I decoded the key file, it does not work.

[neteye@localhost ~]$ ./openssl-3.0.12/apps/openssl ec -in ./cert/testenc.key -text

 read EC key
 Could not read private key from ./cert/testenc.key
 4037D694847F0000:error:1608010C:STORE routines:ossl_store_handle_load_result:unsupported:crypto/store/store_result.c:151:
 unable to load Key

testenc.zip

andylee521 · 2024-01-12T00:53:46Z

Hi t8m,
I double checked my code(3.0.12), I found it be merged pull request #22529.
This code is the root cause for decoding failed the certificate.

[neteye@localhost ~]$ ./openssl-3.0.12/apps/openssl x509 -in cert/testsig.pem -noout -text

Certificate:
Data:
Version: 3 (0x2)
Serial Number: 3 (0x3)
Signature Algorithm: SM2-with-SM3
Issuer: CN = root
Validity
Not Before: Jan 10 00:47:35 2024 GMT
Not After : Jan 10 00:47:35 2029 GMT
Subject: CN = test
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Unable to load Public Key
4017ECD77E7F0000:error:03000072:digital envelope routines:X509_PUBKEY_get0:decode error:crypto/x509/x_pubkey.c:458:
4017ECD77E7F0000:error:03000072:digital envelope routines:X509_PUBKEY_get0:decode error:crypto/x509/x_pubkey.c:458:
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Cert Type:
SSL Client, S/MIME, Object Signing
X509v3 Key Usage:
Digital Signature
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
61:94:56:A4:39:44:25:F5:B8:77:4C:DE:FA:C8:91:0B:FD:50:91:C5
X509v3 Authority Key Identifier:
DirName:/CN=root
serial:01
Signature Algorithm: SM2-with-SM3
Signature Value:
30:46:02:21:00:86:d7:f7:d9:38:10:91:77:cf:18:26:e3:ea:
3f:a9:04:6d:eb:2a:9d:26:d6:f8:e6:a7:95:15:2c:d8:c8:0c:
bb:02:21:00:cc:94:7a:17:4c:2f:8c:ee:96:9a:2e:94:56:b8:
33:92:db:2d:67:a1:ed:c2:3b:5d:92:af:35:a7:0f:70:33:77

jamuir · 2024-01-12T02:38:58Z

I double checked my code(3.0.12), I found it be merged pull request #22529.
This code is the root cause for decoding failed the certificate.

okay. So your 3.0.12 sources are modified (i.e. you are not using the released version).

For comparison, maybe you could try building 3.0.12 from the unmodified sources and attempt SM2 key operations against it.

andylee521 · 2024-01-12T02:54:03Z

I double checked my code(3.0.12), I found it be merged pull request #22529.
This code is the root cause for decoding failed the certificate.

okay. So your 3.0.12 sources are modified (i.e. you are not using the released version).

For comparison, maybe you could try building 3.0.12 from the unmodified sources and attempt SM2 key operations against it.

Em...
For private key, 3.0.12 also does not work，it‘s unmodified.

jamuir · 2024-01-12T03:04:01Z

But I decoded the key file, it does not work.

I am able to reproduce that failure using 3.0.12:

openssl-3.0.12$ LD_LIBRARY_PATH=. apps/openssl ec -in testenc.key -text -noout
read EC key
Could not read private key from testenc.key
40D7F645077F0000:error:1608010C:STORE routines:ossl_store_handle_load_result:unsupported:crypto/store/store_result.c:151:
unable to load Key

However, when I try creating my own SM2 private key, it is parsed successfully:

openssl-3.0.12$ LD_LIBRARY_PATH=. apps/openssl genpkey -algorithm SM2 -out sm2-key.pem
openssl-3.0.12$ LD_LIBRARY_PATH=. apps/openssl ec -in sm2-key.pem -text -noout
read EC key
Private-Key: (256 bit)
priv:
    c9:ba:cd:ae:2b:05:eb:a0:00:bf:e5:21:54:34:de:
    9b:a1:87:20:e8:17:63:68:1e:f9:34:01:3d:f2:b2:
    ba:44
pub:
    04:90:5f:58:3d:53:72:ac:bd:19:0a:1e:a5:a7:80:
    2a:4b:c8:86:44:3d:03:66:3b:03:8d:bf:52:59:b1:
    04:0e:7a:06:9d:d3:b8:51:7e:ee:a0:71:67:20:25:
    95:ba:92:01:2f:7e:55:5a:fa:db:7a:ab:43:f0:1a:
    20:1c:30:db:5a
ASN1 OID: SM2

You can use asn1parse to see that the format of testenc.key is different from sm2-key.pem.

openssl-3.0.12$ LD_LIBRARY_PATH=. apps/openssl asn1parse -in sm2-key.pem 
    0:d=0  hl=3 l= 136 cons: SEQUENCE          
    3:d=1  hl=2 l=   1 prim: INTEGER           :00
    6:d=1  hl=2 l=  20 cons: SEQUENCE          
    8:d=2  hl=2 l=   8 prim: OBJECT            :sm2
   18:d=2  hl=2 l=   8 prim: OBJECT            :sm2
   28:d=1  hl=2 l= 109 prim: OCTET STRING      [HEX DUMP]:306B0201010420C9BACDAE2B05EBA000BFE5215434DE9BA18720E81763681EF934013DF2B2BA44A14403420004905F583D5372ACBD190A1EA5A7802A4BC886443D03663B038DBF5259B1040E7A069DD3B8517EEEA07167202595BA92012F7E555AFADB7AAB43F01A201C30DB5A

openssl-3.0.12$ LD_LIBRARY_PATH=. apps/openssl asn1parse -in testenc.key
    0:d=0  hl=2 l= 119 cons: SEQUENCE          
    2:d=1  hl=2 l=   1 prim: INTEGER           :01
    5:d=1  hl=2 l=  32 prim: OCTET STRING      [HEX DUMP]:F18AD22E520887550A5ACD8C884AE4093A4EA01A08CD2C17DD016CC6ADFCD21F
   39:d=1  hl=2 l=  10 cons: cont [ 0 ]        
   41:d=2  hl=2 l=   8 prim: OBJECT            :sm2
   51:d=1  hl=2 l=  68 cons: cont [ 1 ]        
   53:d=2  hl=2 l=  66 prim: BIT STRING

But, if I parse the octet string starting at offset 30 inside sm2-key.pem, then the output is similar to testenc.key:

openssl-3.0.12$ LD_LIBRARY_PATH=. apps/openssl asn1parse -in sm2-key.pem -offset 30
    0:d=0  hl=2 l= 107 cons: SEQUENCE          
    2:d=1  hl=2 l=   1 prim: INTEGER           :01
    5:d=1  hl=2 l=  32 prim: OCTET STRING      [HEX DUMP]:C9BACDAE2B05EBA000BFE5215434DE9BA18720E81763681EF934013DF2B2BA44
   39:d=1  hl=2 l=  68 cons: cont [ 1 ]        
   41:d=2  hl=2 l=  66 prim: BIT STRING

So the problem is that openssl ec expects testenc.key to have a wrapper / header applied to it but it is missing.

If you add the wrapper / header to testenc.key, then it should be parsed successfully.

jamuir · 2024-01-12T03:38:27Z

If you add the wrapper / header to testenc.key, then it should be parsed successfully.

This is the header you can prepend to the bytes of testenc.key so that openssl ec will parse it:

308194
020100
3014
0608
2a811ccf5501822d
0608
2a811ccf5501822d
0479

So the new key file looks like [header] + [bytes of DER version of testenc.key].

andylee521 · 2024-01-12T05:05:49Z

jamuir,got it, thank you very much.

levitte · 2024-01-12T05:30:47Z

testenc.key looks like this:

So, this is a ECPrivateKey (RFC 5915 - 3. Elliptic Curve Private Key Format) without a PrivateKeyInfo wrapping, what OpenSSL sometimes calls "traditional" form. This is OK.

However, 3.0 OpenSSL types them in PEM with "BEGIN SM2 PRIVATE KEY" when encoding into PEM, while pre-3.0 OpenSSL types them with "BEGIN EC PRIVATE KEY" as seen above... and it's possible that the decoder pays a bit too much attention to the PEM type in this case (encoding difference non withstanding).

levitte · 2024-01-12T05:51:36Z

In #22529, I looked through all SM2 keys we have in the OpenSSL source, and found that they could all be decoded... so it frankly surprised me that we hadn't hit this case. But, looking again, I found that none of them were in "traditional" form, i.e. they were wrapped with a PrivateKeyInfo structure, and had the PEM type "BEGIN PRIVATE KEY", which differs quite a bit from "BEGIN EC PRIVATE KEY", both in form and process.

levitte · 2024-01-12T06:45:37Z

While debugging openssl ec -in ~/Hämtat/testenc.key -text, I had a breakpoint on ossl_store_handle_load_result, and found that it was called a bit early in the process:

(gdb) bt
#0  ossl_store_handle_load_result (params=0x7fffffffcb20, arg=0x7fffffffcf40)
    at ../master/crypto/store/store_result.c:87
#1  0x00007ffff7c8864e in file_load_construct (decoder_inst=0x55555569ae70, 
    params=0x7fffffffcb20, construct_data=0x7fffffffceb0)
    at ../master/providers/implementations/storemgmt/file_store.c:406
#2  0x00007ffff79f7d4a in decoder_process (params=0x7fffffffcb20, 
    arg=0x7fffffffccd0) at ../master/crypto/encode_decode/decoder_lib.c:774
#3  0x00007ffff7c42ea1 in pem2der_decode (vctx=0x5555556a0540, 
    cin=0x5555556a0560, selection=0, data_cb=0x7ffff79f7be1 <decoder_process>, 
    data_cbarg=0x7fffffffccd0, 
    pw_cb=0x7ffff7a83616 <ossl_pw_passphrase_callback_dec>, 
    pw_cbarg=0x55555569ad58)
    at ../master/providers/implementations/encode_decode/decode_pem2der.c:205
#4  0x00007ffff79f8272 in decoder_process (params=0x0, arg=0x7fffffffce00)
    at ../master/crypto/encode_decode/decoder_lib.c:1000
#5  0x00007ffff79f6621 in OSSL_DECODER_from_bio (ctx=0x55555569ad20, 
    in=0x55555569ac90) at ../master/crypto/encode_decode/decoder_lib.c:82
#6  0x00007ffff7c88aa5 in file_load_file (ctx=0x555555693cc0, 
    object_cb=0x7ffff7bc5334 <ossl_store_handle_load_result>, 
    object_cbarg=0x7fffffffcf40, 
    pw_cb=0x7ffff7a83616 <ossl_pw_passphrase_callback_dec>, 
    pw_cbarg=0x555555693eb8)
    at ../master/providers/implementations/storemgmt/file_store.c:532
#7  0x00007ffff7c890ae in file_load (loaderctx=0x555555693cc0, 
    object_cb=0x7ffff7bc5334 <ossl_store_handle_load_result>, 
    object_cbarg=0x7fffffffcf40, 
    pw_cb=0x7ffff7a83616 <ossl_pw_passphrase_callback_dec>, 
    pw_cbarg=0x555555693eb8)
    at ../master/providers/implementations/storemgmt/file_store.c:698
#8  0x00007ffff7bc2201 in OSSL_STORE_load (ctx=0x555555693e70)
    at ../master/crypto/store/store_lib.c:447
#9  0x000055555560e479 in load_key_certs_crls (
    uri=0x7fffffffd92d "/home/levitte/Hämtat/testenc.key", format=0, 
    maybe_stdin=1, pass=0x0, desc=0x555555636f8d "key", quiet=0, 
    ppkey=0x7fffffffd130, ppubkey=0x0, pparams=0x0, pcert=0x0, pcerts=0x0, 
    pcrl=0x0, pcrls=0x0) at ../master/apps/lib/apps.c:1021
#10 0x000055555560d2ac in load_key (
    uri=0x7fffffffd92d "/home/levitte/Hämtat/testenc.key", format=0, 
    may_stdin=1, pass=0x0, e=0x0, desc=0x555555636f8d "key")
    at ../master/apps/lib/apps.c:613
#11 0x00005555555d0ec2 in pkey_main (argc=4, argv=0x7fffffffd500)
    at ../master/apps/pkey.c:218
#12 0x00005555555c913f in do_cmd (prog=0x5555556921f0, argc=4, 
    argv=0x7fffffffd500) at ../master/apps/openssl.c:426
#13 0x00005555555c8d01 in main (argc=4, argv=0x7fffffffd500)
    at ../master/apps/openssl.c:307

... but the real surprise is that it never got called again (i.e. not after the DER→key decoders were called, if they were called at all).

I not yet sure if this is a fundamental issue that causes the decoding failure, but...

jamuir · 2024-01-17T22:46:14Z

@levitte : can this be closed? It seems like we have worked through the original problem.

andylee521 added the issue: bug report The issue was opened to report a bug label Jan 11, 2024

t8m added branch: master Merge to master branch triaged: bug The issue/pr is/fixes a bug branch: 3.0 Merge to openssl-3.0 branch branch: 3.1 Merge to openssl-3.1 branch: 3.2 Merge to openssl-3.2 help wanted and removed issue: bug report The issue was opened to report a bug labels Jan 11, 2024

jamuir removed the branch: master Merge to master branch label Jan 11, 2024

jamuir removed branch: 3.0 Merge to openssl-3.0 branch branch: 3.1 Merge to openssl-3.1 branch: 3.2 Merge to openssl-3.2 labels Jan 11, 2024

jamuir added resolved: not a bug The issue is not considered a bug and removed help wanted labels Jan 12, 2024

nhorman closed this as completed Apr 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

openssl-3.0.12 cannot decode beijing CA certificate。 #23259

openssl-3.0.12 cannot decode beijing CA certificate。 #23259

andylee521 commented Jan 11, 2024

jamuir commented Jan 11, 2024

jamuir commented Jan 11, 2024

jamuir commented Jan 11, 2024

andylee521 commented Jan 12, 2024 •

edited

andylee521 commented Jan 12, 2024 •

edited

jamuir commented Jan 12, 2024

andylee521 commented Jan 12, 2024

jamuir commented Jan 12, 2024 •

edited

jamuir commented Jan 12, 2024 •

edited

andylee521 commented Jan 12, 2024

levitte commented Jan 12, 2024

levitte commented Jan 12, 2024

levitte commented Jan 12, 2024

jamuir commented Jan 17, 2024

openssl-3.0.12 cannot decode beijing CA certificate。 #23259

openssl-3.0.12 cannot decode beijing CA certificate。 #23259

Comments

andylee521 commented Jan 11, 2024

jamuir commented Jan 11, 2024

jamuir commented Jan 11, 2024

jamuir commented Jan 11, 2024

andylee521 commented Jan 12, 2024 • edited

andylee521 commented Jan 12, 2024 • edited

jamuir commented Jan 12, 2024

andylee521 commented Jan 12, 2024

jamuir commented Jan 12, 2024 • edited

jamuir commented Jan 12, 2024 • edited

andylee521 commented Jan 12, 2024

levitte commented Jan 12, 2024

levitte commented Jan 12, 2024

levitte commented Jan 12, 2024

jamuir commented Jan 17, 2024

andylee521 commented Jan 12, 2024 •

edited

andylee521 commented Jan 12, 2024 •

edited

jamuir commented Jan 12, 2024 •

edited

jamuir commented Jan 12, 2024 •

edited