crypto/x509: ParseCertificate ignores all but first value from Subject's []string{} fields

[joemiller]

What version of Go are you using (go version)?

go1.7.4, go1.8rc1

What operating system and processor architecture are you using (go env)?

darwin/amd64

What did you do?

When parsing a certificate with the x509.ParseCertificate() function it ignores all but the first value from attributes in the Subject (pkix.Name) struct that support multiple values.

A cert contains a Subject (pkix.Name) struct. Some of the attributes in this struct support multiple values, represented as []string{}, such as OrganizationalUnit, Locality, etc:

type Name struct {
        Country, Organization, OrganizationalUnit []string
        Locality, Province                        []string
        StreetAddress, PostalCode                 []string
        SerialNumber, CommonName                  string

        Names      []AttributeTypeAndValue
        ExtraNames []AttributeTypeAndValue
}

When x509.ParseCertificate() parses a certificate that has, for example, two OrganizationalUnit (OU) attributes only the first one is parsed and set in the returned x509.Certificate

Note that x509.CreateCertificate() will correctly create certs with multiple values.

Example code and additional commentary demonstrating this issue in this gist: https://gist.github.com/joemiller/c97a52d46cae0a4b38df841db8307fc4

I can move the info from the gist into this issue if that is desirable. I don't know the golang issue etiquette.

What did you expect to see?

x509.ParseCertificate() should return an x509.Certificate containing all of the values for the fields that are of type []string{}

What did you see instead?

I only see the first value for each field.

Discovered during this work: hashicorp/vault#2251

[csstaub]

I tried to reproduce this on go1.8rc1 running on darwin/amd64, seems to work fine for me:

package main

import (
	"crypto/x509"
	"encoding/pem"
	"fmt"
)

const testCertificate = `
-----BEGIN CERTIFICATE-----
MIIDTzCCAjegAwIBAgIJAPd2rUd3n9vNMA0GCSqGSIb3DQEBCwUAMDExDTALBgNV
BAMTBHRlc3QxDzANBgNVBAsTBnZhbHVlMTEPMA0GA1UECxMGdmFsdWUyMB4XDTE3
MDExNDA3MTMwNloXDTE5MTAxMTA3MTMwNlowMTENMAsGA1UEAxMEdGVzdDEPMA0G
A1UECxMGdmFsdWUxMQ8wDQYDVQQLEwZ2YWx1ZTIwggEiMA0GCSqGSIb3DQEBAQUA
A4IBDwAwggEKAoIBAQDOm1c/1c/Ry5/H1WOG09bL/XuCbOhNT62yzyVfGk9dy/oB
HobuV+ow/ctUJ43dgFsJgkJ+w3f/+DLlmRFwoaSbHoVV+5hWdCtBmKCAwWYJ4m0d
bppYGU/IAfGuDSgsSTmQs/MEJ6T5jRdSNIBAQy3ci0stsZsZazGZ+i0TPTQByRvn
F3c1cU9QXKzB52BRU3qm0X+SFvn+uEoabI3Nf1AXWqXye5yybJtWffdc/N5kA7Bz
VqTU314DUxob1LoTbKsv1agGoBfIYMvhsbVkJpSRSKX/X+scYv7J4RrEhPDX+TWJ
3YYjE5dHNgZxqCf1kPKi6ms7iM6u1TBpqshwHaFTAgMBAAGjajBoMAwGA1UdEwEB
/wQCMAAwCwYDVR0PBAQDAgTwMEsGA1UdIwREMEKhNaQzMDExDTALBgNVBAMTBHRl
c3QxDzANBgNVBAsTBnZhbHVlMTEPMA0GA1UECxMGdmFsdWUyggkA93atR3ef280w
DQYJKoZIhvcNAQELBQADggEBAG8MH0plZthQBpdoGK3uyh2mErQ9IHLDi9r2ykLV
h24GdV0UExqL5LAr1l+Nd23KFfYMr3E8nm+PWkqZCOyQTVj7Jby2ETqL+DJWVcJp
ZO9HhgQ2sOvicg2GNIqSGUm1qH3BJwBo3NPGFF/QgBJeuZnJzPaWZrUoriUUDyJ6
/TRm5wRXFMlekt2g3/zpNqeHZSQm4zNijY+qJ+BNZumtPSE6i5093neNjOPhY4yI
n/GWK1XL4ET98qvNHM05gO0cWIe1ZnYEFzyylMPEwWLcluIrBy2aj9xbKR5FKWXb
LDSb0SqDg4UMbsQw3AidGXo5Vluc22KigTrJV9mf1OrUm7Q=
-----END CERTIFICATE-----`

func main() {
	block, _ := pem.Decode([]byte(testCertificate))
	cert, _ := x509.ParseCertificate(block.Bytes)

	fmt.Printf("CN = %v\n", cert.Subject.CommonName)
	fmt.Printf("OU = %v\n", cert.Subject.OrganizationalUnit)
}

Parses and populates both OU values in the embedded certificate:

$ go version
go version go1.8rc1 darwin/amd64
$ go build -o dump main.go && ./dump
CN = test
OU = [value1 value2]

OpenSSL for comparison:

$ openssl x509 -in test-x509v3.crt -noout -text | grep Subject:
        Subject: CN=test, OU=value1, OU=value2

The linked foo.go program (from gist) also seems to parse multiple values correctly:

$ go build -o foo foo.go && ./foo | grep 'ou[12]'
  (string) (len=3) "ou1",
  (string) (len=3) "ou2"
   Value: (string) (len=3) "ou1"
   Value: (string) (len=3) "ou2"

Used grep ou[12] to condense output for readability, but works for all multi-value fields equally.

[joemiller]

With your code I get identical results on 1.7 and 1.8rc:

$ go version
go version go1.8rc1 darwin/amd64
$ go build -o dump f.go && ./dump
CN = test
OU = [value1 value2]

$ go version
go version go1.7.4 darwin/amd64
$ go build -o dump f.go && ./dump
CN = test
OU = [value1 value2]

Using my code from the gist, I get differing results between 1.7 and 1.8:

$ go version
go version go1.7.4 darwin/amd64
$  go build -o foo foo.go && ./foo | grep 'ou[12]'
  (string) (len=3) "ou1"
   Value: (string) (len=3) "ou1"

$ go version
go version go1.8rc1 darwin/amd64
$ go build -o foo foo.go && ./foo | grep 'ou[12]'
  (string) (len=3) "ou1",
  (string) (len=3) "ou2"
   Value: (string) (len=3) "ou1"
   Value: (string) (len=3) "ou2"

I'm a bit confused at this point, but I'll keep poking. Thanks for the feedback.

[joemiller]

Trying a slight variation:

Using your code, but with a cert.pem created by the code in my (foo.go) gist. I get differing behavior on 1.7.4 -vs- 1.8rc1, even though I had identical behavior using your code with your original cert fixture:

package main

import (
        "crypto/x509"
        "encoding/pem"
        "fmt"
)

const testCertificate = `
-----BEGIN CERTIFICATE-----
MIIEVDCCAzygAwIBAgIRALYZF1aMGVFSHyIZJmWmSAQwDQYJKoZIhvcNAQELBQAw
gckxFjAJBgNVBAYTAmMxMAkGA1UEBhMCYzIxHDAMBgNVBAgTBXByb3YxMAwGA1UE
CBMFcHJvdjIxFjAJBgNVBAcTAmwxMAkGA1UEBxMCbDIxHDAMBgNVBAkTBWFkZHIx
MAwGA1UECRMFYWRkcjIxHDAMBgNVBBETBXBvc3QxMAwGA1UEERMFcG9zdDIxIzAO
BgNVBAoTB0FjbWUgQ28wEQYDVQQKEwpzZWNvbmQgb3JnMRgwCgYDVQQLEwNvdTEw
CgYDVQQLEwNvdTIwHhcNMTcwMTE0MTc1NTM3WhcNMTcwMTE0MjA1NTM3WjCByTEW
MAkGA1UEBhMCYzEwCQYDVQQGEwJjMjEcMAwGA1UECBMFcHJvdjEwDAYDVQQIEwVw
cm92MjEWMAkGA1UEBxMCbDEwCQYDVQQHEwJsMjEcMAwGA1UECRMFYWRkcjEwDAYD
VQQJEwVhZGRyMjEcMAwGA1UEERMFcG9zdDEwDAYDVQQREwVwb3N0MjEjMA4GA1UE
ChMHQWNtZSBDbzARBgNVBAoTCnNlY29uZCBvcmcxGDAKBgNVBAsTA291MTAKBgNV
BAsTA291MjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALwmq1o2NSYI
IaR5pqCu6y0PyCrX6pm7V2goeFM44Ydj3fAh9b+oA8gXIdCQcML4nKVkVIg+isLR
KTC8eN/F8zMPm01Gzzog1I4mvahk10HojFaxUEkFD99fg4uWyZDxQWGfGPQA4IKJ
X6vHEgN7CpqnafAjeqoICrc2pigAaHca4Of3Vq6a1lZzMT2eIi3ESxVNukc/vEUo
4azJJ3uUjXBhG2GK6wGKsP5ponZw0jQHj8lny/nAi7XT2NYJ2Q6U1I+7zhHB0Xjz
EEz62ZwrbvxGwGXkrlfBJ+3hdi3ggKyDeIRfgBDU4xqHh6+zfwRKI4D0mARDQhF8
k6VgygbUJO8CAwEAAaM1MDMwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoGCCsG
AQUFBwMBMAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQELBQADggEBAHj1EFJ6Jav2
Xs1KUsDbBQlcbbh4ebTJhSOjEiC1Jg31JQ9A/0WHtSlAWh7KfV51eG2xNFBwUD5x
y53XNR9wuEZARZhgY5ovhr+4fmZHODPTQrAlq/KCc4rf+ecPtYR8Wu75DJZkyWSW
4td4O0+xD/aexqQsqYm4sRyf6d/XzUCgj4wjUCw4hcOT8mnARL8BSnJpbevDOxjL
iiTel4LSrYsoYdtjd2SHNJ5rlQ7fl7ccVN7NBSgxdzskGJZUvllC6phwOIfM141R
3n9tiO6Wb043WYO3exrptKTIz3HxZWhEYZUbkwQ+yU3rt6PQOOTN0HEe7xb1PG2W
VICti8IoCn4=
-----END CERTIFICATE-----
`

func main() {
        block, _ := pem.Decode([]byte(testCertificate))
        cert, _ := x509.ParseCertificate(block.Bytes)

        fmt.Printf("CN = %v\n", cert.Subject.CommonName)
        fmt.Printf("OU = %v\n", cert.Subject.OrganizationalUnit)
}

$ go version
go version go1.7.4 darwin/amd64
$ go build -o dump main.go && ./dump
CN =
OU = [ou1]

$ go version
go version go1.8rc1 darwin/amd64
$ go build -o dump main.go && ./dump
CN =
OU = [ou1 ou2]

I'm confused as to why the cert fixture in your code works as expected in both 1.7.4 and 1.8 for me, while the other cert only works as expected in 1.8. In any case, I'm getting better results from 1.8 in general at this time, so possibly a non-issue in 1.8+ which is good.

[csstaub]

Can confirm, seeing the same difference in behavior (using your test cert, but not mine) in 1.7.1 vs. 1.8rc1.

[csstaub]

It appears there's a difference in the way the subjects are encoded.

From the working cert (output from dumpasn1):

 56  15:       SET {
 58  13:         SEQUENCE {
 60   3:           OBJECT IDENTIFIER organizationalUnitName (2 5 4 11)
 65   6:           PrintableString 'value1'
       :           }
       :         }
 73  15:       SET {
 75  13:         SEQUENCE {
 77   3:           OBJECT IDENTIFIER organizationalUnitName (2 5 4 11)
 82   6:           PrintableString 'value2'
       :           }
       :         }
       :       }

From the broken cert:

 461   24:       SET {
 463   10:         SEQUENCE {
 465    3:           OBJECT IDENTIFIER organizationalUnitName (2 5 4 11)
 470    3:           PrintableString 'ou1'
         :           }
 475   10:         SEQUENCE {
 477    3:           OBJECT IDENTIFIER organizationalUnitName (2 5 4 11)
 482    3:           PrintableString 'ou2'
         :           }
         :         }

For reference, my test cert was generated with OpenSSL 0.9.8zh (which ships in macOS Sierra).

[agl]

One of the sharp edges of X.509 is that the name structure isn't a mapping from OIDs to a list of values, it's a sequence of sets of pairs. I.e. something like [][]struct{key OID, value string}. (I think the original motivation was to try and identify which values of the multiset were alternatives, and which were needed for a unique identification. These days, nobody cares and that over-engineering is now just a source of confusion.)

There was a bug, fixed in 809a1de, that meant that code previously only looked at first element of these sublists. So the difference between the certificates that work and fail with 1.7 is that the multiple values are either duplicated at the top-level, or the sublevel. You can see this with openssl asn1parse -i.

[agl]

(Since this should be fixed in 1.8, I think this can be closed.)