crypto/tls: handshake error with custom/local CA (bad record MAC & certificate signed by unknown authority)

[gopherbot]

by hanks_j@ligo-wa.caltech.edu:

What steps will reproduce the problem?
If possible, include a link to a program on play.golang.org.

1. Setup a web-server serving https with a certificate from a local CA

2. Point the program listed at http://play.golang.org/p/iKScTFUQl3 at the server

3. The program is unable to complete the TLS handshake.

What is the expected output?

The program should complete a TLS handshake and exit without errors.

What do you see instead?

Two error conditions:
 1. if certificate verification is enabled, it states that the certificate is signed by an unknown authority
 2. if certificate verification is disabled, it states that there is a bad record MAC.

I have verified that the certificate chain is complete, that the certificate and key are
the correct pair.

Which compiler are you using (5g, 6g, 8g, gccgo)?

gc

Which operating system are you using?

Ubuntu 10.04 amd 64bit
Debian 6 amd 64bit

Which version are you using?  (run 'go version')

Tested on 1.0.2, 1.0.3, tip

Please provide any additional information below.

This could be a problem with my CA setup.  However the CA and the test web server
satisfy both gnutls and openssl.  Both report the certificate chain as being valid.  I
can make the connection with anything else.

I have attached a sample certificate, key, and ca certificate that I tested against.  It
is a horribly weak key.  However key length does not seem to change the outcome.  I have
tested this with 512 bit and 2048 bit rsa keys.

I ran into this issue while attempting to consume a https resource in a program I am
working on.

Attachments:

1. server1_cert.pem (1018 bytes)
2. server1_key.pem (493 bytes)
3. ca-cert.pem (855 bytes)

[rsc]

Comment 2:

Labels changed: added priority-later, removed priority-triage.

Status changed to Accepted.

[agl]

Comment 3:

What's the webserver? The most likely explanation is a problem with completing the TLS
handshake.

[agl]

Comment 4:

What's the webserver? The most likely explanation is a problem with completing the TLS
handshake.

[agl]

Comment 5:

What's the webserver? The most likely explanation is a problem with completing the TLS
handshake.

[gopherbot]

Comment 6 by hanks_j@ligo-wa.caltech.edu:

I have tested with a variety of servers and clients.
Servers:
1. Apache 2.2.x
2. NginX 1.1.19
3. openssl s_server
4. gnutls-serv
Clients:
1. Firefox
2. openssl s_client
3. gnutls-cli
4. the tls_error.go file listed above
With each server and clients 1,2, & 3 I can do a successful TLS handshake (specifying
TLSv1 where possible) with verification of the certificate, providing that I specify
ca-cert.pem file as the trusted CA list.
When I use the test code I have in tls_error.go I cannot finish a TLS handshake.  With
certificate verification enabled it states that it cannot verify the certificate.  With
verification disabled it has a bad record MAC.
I have run a number of screen captures with a variety of server/client combinations. 
Each combination used the same set of certificates and keys.
The Apache*log.txt files may be the most interesting ones, as they show a distinctly
different output between using the go code vs openssl s_client and firefox.

Attachments:

1. Apache-Firefox-logs.txt (10064 bytes)
2. Apache-OpenSSL-with-logs.txt (12185 bytes)
3. Apache-go-with-logs (5917 bytes)
4. Apache-gnutls-cli-no-cacert.txt (1306 bytes)
5. Apache-gnutls-cli-w-cacert.txt (1326 bytes)
6. Apache-go-test-no-verify.txt (1010 bytes)
7. Apache-go-test-w-verify.txt (1026 bytes)
8. Apache-openssl-s_client-no-cacert.txt (5248 bytes)
9. Apache-openssl-s_client-w-cacert.txt (4963 bytes)
10. GnuTLS-serv-tests.txt (9873 bytes)
11. NginX-gnutls-cli-no-cacert.txt (1308 bytes)
12. NginX-gnutls-cli-w-cacert.txt (1328 bytes)
13. NginX-go-test-no-verify.txt (1012 bytes)
14. NginX-go-test-w-verify.txt (1028 bytes)
15. NginX-openssl-s_client-no-cacert.txt (5258 bytes)

[gopherbot]

Comment 7 by hanks_j@ligo-wa.caltech.edu:

I also built go from tip today and ran this on OS X 10.8.
More of the same.  Logs attached.  But it the same as above.

Attachments:

1. OpenSSL-s_serv-OSX-test.txt (5565 bytes)

[gopherbot]

Comment 8 by hanks_j@ligo-wa.caltech.edu:

The code I have listed above is a smaller version of this test.  I have a version that
uses an instrumented version of the crypto/rsa crypto/x509 code (added fmt.Printlns).  
That shows that x509/verify.go fails to build a chain of trust between the server
certificate and the CA list in buildChains.
x509/cert_poo.go findVerifiedParents comes up with a candidate certificate, but fails to
verify the signature.
I'm perplexed at the error.  I generated the certificates, I know that server1_cert, is
signed by ca-cert.  I think that three crypto implementations agree with me (openssl,
gnutls, and the netscape/mozilla code).  But I won't rule out errors on my part.

[agl]

Comment 9:

Firstly, sorry about the duplicate comments - codeside had a meltdown this afternoon.
I think there are two problems here: one with the certificate and one, perhaps, with the
TLS library.
Can you expose any of these problematic servers to the public Internet so that I can
poke it? The TLS issue (bad MAC) is more interesting. I can load the chain from #1
tomorrow and figure out why it's not working.

[agl]

Comment 10:

So, with the first issue (the certificate problem), the issue is that the software that
you used to generate the certificates (both) is broken: the public keys are negative
numbers. ASN.1 requires that positive numbers with the most-significant-bit set be
prefixed with 0x00, and these certs don't do that. 
That took a while to track down but I'll change the code to return a more helpful error
message in the future.
I think you might have a second problem (the bad MAC error) so please get back to me if
that's still happening with valid certs.

[gopherbot]

Comment 11 by hanks_j@ligo-wa.caltech.edu:

Thank you for looking at that, sorry for taking your time on something that openssl,
gnutls, ... should have pointed out (and not generated in the first place).
At the least a helpful error message, on load or use of a certificate would be nice.
Well, I've got to go back and re-key things.  After I change my tooling to catch this.

[agl]

Comment 12:

This issue was updated by revision 5c659d7.

Return more helpful errors when RSA parameters are negative or zero.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7228072

[agl]

Comment 13:

What software did you use to generate these certs?
If everything works after fixing the certs, please poke this bug and I'll close it.

[gopherbot]

Comment 14 by hanks_j@ligo-wa.caltech.edu:

I generated the certificates (both ca-cert and server1_cert) with gnutls's certtool,
version 2.8.5 as shipped with Ubuntu 10.04 LTS latest patches.
I will generate a new CA and certificate and test, after reviewing procedure and output.
 It may take a day or two to fit it into my schedule.

[gopherbot]

Comment 15 by hanks_j@ligo-wa.caltech.edu:

Agl,
Thanks.  I had some hurry up and wait time.  So I build a new CA and tested.  I am able
to do a TLS handshake against new certificates.
When I run the code against the older (bad) certs it fails to load the CA certificate
file.

[agl]

Comment 17:

Status changed to WorkingAsIntended.