Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

X.509 CertificateBuilder does not ensure issuer bytes precisely match the issuer's subject #3515

Closed
thinkwelltwd opened this issue Apr 22, 2017 · 12 comments · Fixed by #3896
Closed

X.509 CertificateBuilder does not ensure issuer bytes precisely match the issuer's subject #3515

thinkwelltwd opened this issue Apr 22, 2017 · 12 comments · Fixed by #3896
Labels
x509
Milestone
Twenty first release

Comments

@thinkwelltwd
Copy link

I posted my problem about this on StackOverflow several months ago, with no reply. After duking it out for awhile, I just stayed on 1.0.1 which generated signed certificates that worked as expected. Here's a snippet that works in 1.0.1, but exhibits the problem described in my SO post on newer versions of Cryptography.

Not a long-term solution as I can't just stay on 1.0.1, and it's time to get to the bottom of the issue. Eventually, I got this script snippet to generate certificates that worked on Windows -- but Linux Mint or OS X still had the old problem of not detecting the Certification path. Also, Firefox on Windows doesn't detect the Certification path.

Being now at my wits end, I decided to use another crypto library, and this snippet using pyopenssl creates and signs certs that work just perfect on all OSs. In a way my problem is solved, except that I'd rather not introduce a new dependency that already depends on Cryptography, which has a nicer API.

So I thought I'd file an issue because it seems like there's a bug. If not a bug, then no clear way in the docs to sign certificates, and if that could save the next guy 25 hours of his life, then it would help recover some of my effort.

Any assistance "Signing certificates For Humans Using Cryptography" would be much appreciated.
Thanks for the excellent work on the Cryptography module!
@alex alex added the x509 label Apr 22, 2017
@alex
Copy link
Member

alex commented Apr 22, 2017 •
edited

So, if I can restate this problem. Using this code https://code.compassfoundation.io/snippets/1, you get a leaf certificate that is trusted when you run the code under cryptogprahy 1.0, but using 1.7.1, you get a certificate which isn't trusted. Is that right?

What OS are you running the code on?

Can you post example certificates that are generated by 1.0 and 1.7.1?

@thinkwelltwd
Copy link
Author

thinkwelltwd commented Apr 22, 2017 •
edited

What OS are you running the code on?

dave@minty ~ $ cat /etc/upstream-release/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04 LTS"

dave@minty ~ $ openssl version
OpenSSL 1.0.1f 6 Jan 2014

The machine that generated the root CA (based on Redhat 6)

OpenSSL 1.0.1e-fips 11 Feb 2013
[root@system ~]# cat /etc/redhat-release
ClearOS Community release 6.8.0 (Final)
[root@system ~]# openssl version
OpenSSL 1.0.1e-fips 11 Feb 2013

Here are some certs for your review certs.zip Your request for example certs led me to do something I should have thought to do before now. I'm now no longer sure if this is a cryptography version problem, or what is the real issue.

I didn't want to post certs created by our real CA, so I regenerated a new root CA, on my Linux Mint 17 machine where I'm signing the certs. With the new CA, both 1.0.1 and 1.8.1 script snippets generated correct SSL Certificates. I was dumbfounded...so I experimented some more, because the Linux Mint box wasn't the source of our company CA.

  1. Created a CA on the debian-based machine ca_generated_in_debian
  2. Created a CA on the redhat-based machine ca_generated_in_redhat. This is the machine that produced our company CA. Our main platform is Redhat-based so that's of most concern.

Then I generated signed certs on Debian & Redhat machines, using both of the CAs mentioned above, with pyopenssl & cryptography 1.0.1 & 1.7.1 on Redhat and pyopenssl & cryptography 1.0.1 & 1.8.1

Summarized Results
The pyopenssl snippet generated certs with correct Certification Paths every possible time, using both CAs on both machines.

Cryptography 1.0.1 on Redhat generates correct Certification Path when using ca_generated_in_redhat
Cryptography 1.0.1 on Redhat generates INCORRECT Certification Path when using ca_generated_in_debian

Cryptography 1.7.1 on Redhat generates INCORRECT Certification Path when using ca_generated_in_redhat
Cryptography 1.7.1 on Redhat generates correct Certification Path when using ca_generated_in_debian

Please doublecheck my results; my head is spinning. I included the certs, so check them for yourself.

Why can pyopenssl get it right every time? I didn't look at the code, but perhaps the signing of the cert doesn't even involve cryptography, and just calls openssl bindings directly?

@alex
Copy link
Member

alex commented Apr 22, 2017

To make sure I follow, using the certs in certs/debian_generated_ca, debianca_signed_on_redhat-1.0.1.pem works, but debianca_signed_on_redhat-1.7.1.der doesn't?

Diffing OpenSSL's representation of them looks entirely innocous:

--- /dev/fd/63	2017-04-22 18:04:35.000000000 -0400
+++ /dev/fd/62	2017-04-22 18:04:35.000000000 -0400
@@ -2,52 +2,52 @@
     Data:
         Version: 3 (0x2)
         Serial Number:
-            a5:9e:4e:77:64:0f:45:02:83:7e:d9:bd:a3:2b:7c:c6
+            b8:96:3c:b7:2a:65:46:90:97:a1:91:9b:6a:9d:3b:61
         Signature Algorithm: sha256WithRSAEncryption
         Issuer: C=US, ST=IL, L=Decatur, O=The Chocolatiers, OU=Sweet and Sour, CN=a.vat.of.chocolate/emailAddress=c@vat.com
         Validity
-            Not Before: Apr 21 17:32:05 2017 GMT
-            Not After : Apr 20 17:32:05 2027 GMT
+            Not Before: Apr 21 17:30:50 2017 GMT
+            Not After : Apr 20 17:30:50 2027 GMT
         Subject: C=US, ST=TX, L=MailBox, O=The Company, CN=thecompany.com/emailAddress=sysadmin@thecompany.com
         Subject Public Key Info:
             Public Key Algorithm: rsaEncryption
             RSA Public Key: (2048 bit)
                 Modulus (2048 bit):
-                    00:9e:8b:9b:0f:8a:bc:a5:1e:1f:1b:e6:56:e7:81:
-                    11:53:ca:4a:75:79:f7:ec:2d:a4:45:3c:cd:a5:4b:
-                    5d:4f:3c:90:5b:05:e3:da:48:28:02:9b:f1:bc:39:
-                    1a:2c:ba:1d:11:6b:9f:96:75:97:89:c4:74:51:ce:
-                    62:5b:f5:78:86:c4:d7:d2:b3:99:09:ae:ce:ac:9d:
-                    6f:bd:40:fb:aa:ae:dd:bc:44:02:0e:f7:22:7a:bc:
-                    c5:b3:7a:a8:0d:4a:12:2c:eb:3e:1e:ef:eb:e5:7b:
-                    a8:73:aa:b2:e0:39:d8:c2:07:f2:ca:8c:64:bc:7d:
-                    54:ae:72:e5:43:95:a4:40:65:bf:30:46:fc:77:e3:
-                    69:c0:9a:4f:59:fe:43:24:11:41:82:25:91:6d:18:
-                    f3:7b:09:f2:b7:d1:68:1a:ca:5e:7c:80:83:2c:a0:
-                    6f:09:79:f3:7c:71:04:7c:c6:a0:73:2e:ab:ca:e0:
-                    17:60:3e:ea:0f:b0:ce:8f:1a:ea:a4:a9:7b:fd:43:
-                    8c:f8:84:4a:13:e1:83:77:12:3c:8a:33:46:1a:0f:
-                    20:24:d4:52:b3:d9:d8:75:cc:82:3a:7f:27:3f:69:
-                    4f:55:40:be:1e:a1:d8:78:e9:a5:3e:02:40:8c:16:
-                    11:34:e4:bf:4d:0d:1a:01:08:c9:61:f6:7e:9c:b0:
-                    30:fb
+                    00:ca:4c:75:3a:00:18:3b:9c:3f:13:97:8e:90:55:
+                    84:ca:c7:7b:54:14:c0:14:e5:f3:1d:29:e1:37:cd:
+                    5a:04:06:b5:68:c6:74:63:57:c4:0d:15:1b:31:db:
+                    26:c7:75:ae:af:f9:19:c2:96:8a:96:e4:2a:46:f8:
+                    39:79:9e:9f:ca:5b:1e:c0:aa:ce:1e:6f:1b:61:ef:
+                    87:d5:2e:5d:3e:da:f0:ac:ad:03:3c:95:47:0a:96:
+                    92:5d:f0:d6:24:41:7a:96:7e:3e:01:d6:a3:09:fa:
+                    6a:1c:19:ab:79:38:3e:0f:d8:70:c1:05:76:3f:e7:
+                    42:b3:b4:96:52:77:76:ab:91:fe:ee:25:17:0d:a0:
+                    d3:2e:43:85:e7:04:8f:ba:4c:78:0e:0b:fe:a4:ad:
+                    69:54:38:1f:7a:d0:e9:8c:0c:7c:22:ae:e2:e6:ba:
+                    2c:6c:2e:a3:3b:61:25:2c:22:80:07:41:72:f1:70:
+                    61:b3:0b:20:1b:4d:62:5e:db:81:62:7f:e6:8c:65:
+                    ea:4c:36:5b:30:09:9e:f5:0f:81:55:60:35:9e:5b:
+                    62:e8:44:fd:57:cc:59:68:b3:5c:74:0b:eb:42:51:
+                    70:d9:34:7b:71:33:2f:d6:0c:d7:ce:a6:9e:20:ad:
+                    76:db:52:2a:b5:87:b3:e7:5d:92:81:40:58:2b:a9:
+                    77:6b
                 Exponent: 65537 (0x10001)
         X509v3 extensions:
             X509v3 Subject Alternative Name: 
                 DNS:thinkwell.poweredbyclear.com, DNS:log.cabin
     Signature Algorithm: sha256WithRSAEncryption
-        60:8f:28:20:b2:ce:e3:7d:dd:ff:2d:40:ac:5a:9e:98:96:b1:
-        d1:45:b2:cc:6c:0d:39:e2:b7:e2:97:76:00:25:f3:30:b8:de:
-        d5:0b:35:5e:19:fa:ce:a8:5b:cd:ec:26:4c:13:ee:72:25:2c:
-        d8:54:04:1a:7b:2d:bc:c4:d2:d6:db:60:5b:c8:e1:ca:48:a0:
-        6d:52:e4:9d:fe:9c:37:75:2b:0c:a7:be:b3:82:87:3e:4b:c5:
-        c1:fe:71:90:ec:e0:b2:03:17:41:a7:53:51:c2:a7:99:c4:bd:
-        f3:d9:39:c8:8d:66:9b:fa:2d:c5:5d:ff:b7:d2:c0:84:97:15:
-        25:1e:82:c8:21:cc:1c:41:ef:09:85:87:94:33:dc:13:cd:ea:
-        0d:8a:0e:4f:6b:56:50:b4:24:44:06:75:7b:2b:ff:13:02:63:
-        67:ac:90:c5:d1:96:76:ed:6b:a1:44:16:7b:b9:0f:2a:d6:90:
-        40:a4:b0:36:af:68:b6:16:2f:69:ad:14:85:46:42:d3:26:a5:
-        3b:bc:ca:36:46:60:c7:cc:95:39:59:ae:07:9e:59:e1:c3:1b:
-        47:86:2e:01:48:1c:3f:1f:52:1b:12:0f:ff:b4:72:6c:ef:cd:
-        f9:2e:3a:c2:30:3c:dd:6b:3d:ef:16:b5:c9:bc:36:d0:33:fd:
-        cc:a9:5e:43
+        0d:a4:ed:fc:84:16:b4:13:ac:a9:49:23:35:b5:77:3a:6a:cb:
+        a2:89:b2:90:bd:08:f7:2c:19:51:f9:29:cb:2d:70:8f:4b:63:
+        76:c4:03:91:34:e4:49:03:a5:f1:29:0d:28:e4:5a:16:09:cd:
+        7e:6e:0c:1d:2b:54:9c:95:79:76:ac:46:07:59:c9:46:d4:1a:
+        75:2a:38:f6:f8:97:04:3f:68:86:0e:03:90:04:30:95:18:e4:
+        23:4b:20:4e:fb:0b:3d:05:ce:e3:7e:dd:1b:d0:e4:4c:af:cf:
+        ae:82:5f:ab:62:75:e9:21:65:00:83:1e:3c:41:c0:77:e4:f4:
+        38:9a:d4:a2:6f:93:a9:e1:e2:2a:f4:71:60:f1:6f:d9:dd:1e:
+        67:0a:b9:39:9f:eb:ef:49:de:86:ea:62:bc:dd:69:6c:a2:92:
+        3d:40:71:63:08:72:36:64:98:c6:7c:d7:ef:be:09:f9:53:c0:
+        8b:9b:75:2e:ad:e0:d9:de:8a:67:fc:2b:85:ad:9b:3a:da:94:
+        76:6e:a6:95:36:a9:84:d7:36:44:86:aa:b3:a0:1b:c0:09:db:
+        e8:77:50:f8:98:1f:22:09:56:8a:ba:4d:81:be:ce:23:d6:76:
+        94:e5:e9:3c:67:ae:d4:11:39:af:11:b7:75:0c:be:26:f6:ee:
+        95:ef:a5:2e

To isolate further, can you change your script slightly so that each certificate generated uses the same private/public key, and post the two certs -- hopefully one of which fails and one of which succeeds?

@alex
Copy link
Member

alex commented Apr 22, 2017

Actually, nevermind, it's easy enough to see the delta even with the different keys:

--- /dev/fd/63	2017-04-22 18:16:56.000000000 -0400
+++ /dev/fd/62	2017-04-22 18:16:56.000000000 -0400
@@ -2,7 +2,7 @@
     4:d=1  hl=4 l= 709 cons:  SEQUENCE          
     8:d=2  hl=2 l=   3 cons:   cont [ 0 ]        
    10:d=3  hl=2 l=   1 prim:    INTEGER           :02
-   13:d=2  hl=2 l=  17 prim:   INTEGER           :A59E4E77640F4502837ED9BDA32B7CC6
+   13:d=2  hl=2 l=  17 prim:   INTEGER           :B8963CB72A65469097A1919B6A9D3B61
    32:d=2  hl=2 l=  13 cons:   SEQUENCE          
    34:d=3  hl=2 l=   9 prim:    OBJECT            :sha256WithRSAEncryption
    45:d=3  hl=2 l=   0 prim:    NULL              
@@ -14,30 +14,30 @@
    63:d=3  hl=2 l=  11 cons:    SET               
    65:d=4  hl=2 l=   9 cons:     SEQUENCE          
    67:d=5  hl=2 l=   3 prim:      OBJECT            :stateOrProvinceName
-   72:d=5  hl=2 l=   2 prim:      PRINTABLESTRING   :IL
+   72:d=5  hl=2 l=   2 prim:      UTF8STRING        :IL
    76:d=3  hl=2 l=  16 cons:    SET               
    78:d=4  hl=2 l=  14 cons:     SEQUENCE          
    80:d=5  hl=2 l=   3 prim:      OBJECT            :localityName
-   85:d=5  hl=2 l=   7 prim:      PRINTABLESTRING   :Decatur
+   85:d=5  hl=2 l=   7 prim:      UTF8STRING        :Decatur
    94:d=3  hl=2 l=  25 cons:    SET               
    96:d=4  hl=2 l=  23 cons:     SEQUENCE          
    98:d=5  hl=2 l=   3 prim:      OBJECT            :organizationName
-  103:d=5  hl=2 l=  16 prim:      PRINTABLESTRING   :The Chocolatiers
+  103:d=5  hl=2 l=  16 prim:      UTF8STRING        :The Chocolatiers
   121:d=3  hl=2 l=  23 cons:    SET               
   123:d=4  hl=2 l=  21 cons:     SEQUENCE          
   125:d=5  hl=2 l=   3 prim:      OBJECT            :organizationalUnitName
-  130:d=5  hl=2 l=  14 prim:      PRINTABLESTRING   :Sweet and Sour
+  130:d=5  hl=2 l=  14 prim:      UTF8STRING        :Sweet and Sour
   146:d=3  hl=2 l=  27 cons:    SET               
   148:d=4  hl=2 l=  25 cons:     SEQUENCE          
   150:d=5  hl=2 l=   3 prim:      OBJECT            :commonName
-  155:d=5  hl=2 l=  18 prim:      PRINTABLESTRING   :a.vat.of.chocolate
+  155:d=5  hl=2 l=  18 prim:      UTF8STRING        :a.vat.of.chocolate
   175:d=3  hl=2 l=  24 cons:    SET               
   177:d=4  hl=2 l=  22 cons:     SEQUENCE          
   179:d=5  hl=2 l=   9 prim:      OBJECT            :emailAddress
   190:d=5  hl=2 l=   9 prim:      IA5STRING         :c@vat.com
   201:d=2  hl=2 l=  30 cons:   SEQUENCE          
-  203:d=3  hl=2 l=  13 prim:    UTCTIME           :170421173205Z
-  218:d=3  hl=2 l=  13 prim:    UTCTIME           :270420173205Z
+  203:d=3  hl=2 l=  13 prim:    UTCTIME           :170421173050Z
+  218:d=3  hl=2 l=  13 prim:    UTCTIME           :270420173050Z
   233:d=2  hl=3 l= 131 cons:   SEQUENCE          
   236:d=3  hl=2 l=  11 cons:    SET               
   238:d=4  hl=2 l=   9 cons:     SEQUENCE          
@@ -46,19 +46,19 @@
   249:d=3  hl=2 l=  11 cons:    SET               
   251:d=4  hl=2 l=   9 cons:     SEQUENCE          
   253:d=5  hl=2 l=   3 prim:      OBJECT            :stateOrProvinceName
-  258:d=5  hl=2 l=   2 prim:      PRINTABLESTRING   :TX
+  258:d=5  hl=2 l=   2 prim:      UTF8STRING        :TX
   262:d=3  hl=2 l=  16 cons:    SET               
   264:d=4  hl=2 l=  14 cons:     SEQUENCE          
   266:d=5  hl=2 l=   3 prim:      OBJECT            :localityName
-  271:d=5  hl=2 l=   7 prim:      PRINTABLESTRING   :MailBox
+  271:d=5  hl=2 l=   7 prim:      UTF8STRING        :MailBox
   280:d=3  hl=2 l=  20 cons:    SET               
   282:d=4  hl=2 l=  18 cons:     SEQUENCE          
   284:d=5  hl=2 l=   3 prim:      OBJECT            :organizationName
-  289:d=5  hl=2 l=  11 prim:      PRINTABLESTRING   :The Company
+  289:d=5  hl=2 l=  11 prim:      UTF8STRING        :The Company
   302:d=3  hl=2 l=  23 cons:    SET               
   304:d=4  hl=2 l=  21 cons:     SEQUENCE          
   306:d=5  hl=2 l=   3 prim:      OBJECT            :commonName
-  311:d=5  hl=2 l=  14 prim:      PRINTABLESTRING   :thecompany.com
+  311:d=5  hl=2 l=  14 prim:      UTF8STRING        :thecompany.com
   327:d=3  hl=2 l=  38 cons:    SET               
   329:d=4  hl=2 l=  36 cons:     SEQUENCE          
   331:d=5  hl=2 l=   9 prim:      OBJECT            :emailAddress

@alex
Copy link
Member

alex commented Apr 22, 2017

Can you say a bit more about what platform fails to verify the certs?

@alex
Copy link
Member

alex commented Apr 22, 2017

Hypothesis about the problem: The bytes between the CA and the issued certs' issuer/subject are not identical, since one is using UTF8STRING and the other is PRINTABLESTRING. /cc @reaperhulk

@thinkwelltwd
Copy link
Author

Can you say a bit more about what platform fails to verify the certs?

It's used to sign certs for network UTMs; a kind of bespoke letsencrypt. The OSs in use are Redhat-based 5/6/7. On Redhat 5, OpenSSL upgraded to 1.0.2j.

Hypothesis about the problem: The bytes between the CA and the issued certs' issuer/subject are not identical, since one is using UTF8STRING and the other is PRINTABLESTRING. /cc @reaperhulk

That's plausible, because the issuer information is clearly visible in the cert viewers of every platform. Just somehow the Certification Path can't be pieced together. Let me know if there's anything else I can do. Will be out the rest of the weekend; I don't work on Sunday...

@alex
Copy link
Member

alex commented Apr 23, 2017 via email

@thinkwelltwd
Copy link
Author

I should have mentioned that I generated the CA with openssl, and not with cryptography.

openssl req -new -newkey rsa:2048 -sha256 -days 365 -nodes -x509 -extensions v3_ca -keyout ca_generated_in_redhat.pem -out ca_generated_in_redhat.pem

Some cert validators require that issuer.subject and subject.issuer are byte-for-byte identical to form a path.

I'm assuming that it's on the cryptography library to be doing this? Or is there anything else that I should be trying, or providing?

@alex
Copy link
Member

alex commented Apr 26, 2017

Yeah, this is a probably a bug in cryptography, but it's not clear how to fix it with our current API.

@alex alex changed the title Signed Certificates with incorrect certification chain X.509 CertificateBuilder does not ensure issuer bytes precisely match the issuer's subject May 25, 2017
@someshi
Copy link

someshi commented Jun 19, 2017 •
edited

The following excerpt could probably aid in modifying x509.Name to fix this issue.

Refer page 8 of https://www.ietf.org/rfc/rfc4514.txt

5.2. Use of Distinguished Names in Security Applications

The transformations of an AttributeValue value from its X.501 form to
an LDAP string representation are not always reversible back to the
same BER (Basic Encoding Rules) or DER (Distinguished Encoding Rules)
form. An example of a situation that requires the DER form of a
distinguished name is the verification of an X.509 certificate.

For example, a distinguished name consisting of one RDN with one AVA,
in which the type is commonName and the value is of the TeletexString
choice with the letters 'Sam', would be represented in LDAP as the
string <CN=Sam>. Another distinguished name in which the value is
still 'Sam', but is of the PrintableString choice, would have the
same representation <CN=Sam>.

Applications that require the reconstruction of the DER form of the
value SHOULD NOT use the string representation of attribute syntaxes
when converting a distinguished name to the LDAP format. Instead,
they SHOULD use the hexadecimal form prefixed by the number sign ('#'
U+0023) as described in the first paragraph of Section 2.4.

To elaborate
builder = X509.CertificateBuilder().issuer_name(x509.Name([rdn for rdn in ca_crt.subject]))
should be able to carry over the encoding information of the original RDN followed by
_encode_name_entry(backend, attribute) being able to use that information.

@reaperhulk
Copy link
Member

Another example of this issue being problematic: https://lists.debian.org/debian-devel/2017/09/msg00101.html

@reaperhulk reaperhulk mentioned this issue Sep 7, 2017
Merged
@reaperhulk reaperhulk added this to the Twenty first release milestone Sep 7, 2017
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
x509
Milestone
Twenty first release
Development

Successfully merging a pull request may close this issue.

both parse and encode the ASN1 string type for Name attributes reaperhulk/cryptography
4 participants
@alex @reaperhulk @someshi @thinkwelltwd