bug: etcd crashes when reading CA key file

[ruphin]

I am trying to set up an etcd cluster with server-to-server verification according to the Security Guide: https://github.com/coreos/etcd/blob/master/Documentation/security.md
However, etcd is unable to start because of a parsing error in my root CA certificate file.

This is what I find in the logfile with a CA file generated by etcd-ca (https://github.com/coreos/etcd-ca)

2015/04/16 19:20:52 etcd: asn1: structure error: tags don't match (16 vs {class:0 tag:2 length:1 isCompound:false}) {optional:false explicit:false application:false defaultValue:<nil> tag:<nil> stringType:0 set:false omitEmpty:false} tbsCertificate @2

I also tried generating my own CA file with OpenSSL with these instructions: http://www.g-loaded.eu/2005/11/10/be-your-own-ca/ as recommended by the etcd security guide, but the same thing happens:

2015/04/16 18:57:45 etcd: asn1: structure error: tags don't match (16 vs {class:1 tag:11330 length:44 isCompound:false}) {optional:false explicit:false application:false defaultValue:<nil> tag:<nil> stringType:0 set:false omitEmpty:false} certificate @4

It seems to be an issue in the go crypto library: golang/go#6650

However, I can't imagine the security features in etcd exist without anyone ever getting them to work, so there must be a workaround or way to make things work. I am on Ubuntu 14.04. I can reproduce the problem by following the etcd security guide. I am willing to test anything or try any suggestions anyone may have.

Thank you

[yichengq]

@ruphin Could you give command line that you use? Why do you think it is a CA file problem?

My guess is that the error may imply that your private key is encrypted instead of plain. Try etcd-ca export --insecure xxx instead?

[ruphin]

I know for sure that it is the CA file, because if I do not set the ETCD_PEER_CA_FILE env variable etcd starts normally.

I use an upstart script that looks something like this (irrelevant lines are omitted):

env ETCD_DATA_DIR=/opt/etcd
env ETCD_NAME=<HOSTNAME>
env ETCD_PEER_CERT_FILE=/opt/etcd-certificates/peer.crt
env ETCD_PEER_KEY_FILE=/opt/etcd-certificates/peer.key
env ETCD_PEER_CA_FILE=/opt/etcd-certificates/ca.key

exec /usr/bin/etcd

If ca.key is an encrypted certificate key, etcd fails to start.
Log output: https://gist.github.com/ruphin/933933706248b63e62c8

If ca.key is an unencrypted certificate key, etcd fails to start as well.
Log output: https://gist.github.com/ruphin/abaa8abdf52df4d5d877

If I comment out out the ETCD_PEER_CA_FILE env, etcd starts normally.
Log output: https://gist.github.com/ruphin/28f9e16f5d45a188f928

I used the insecure export feature like you suggested to create an unencrypted file. I also tried creating CA certificates myself using OpenSSL. The certificates seem fine, only etcd crashes when reading them. I spent several hours trying every possible way to configure and generate certificates, with and without passwords, exporting them, unencrypting them manually, but etcd did not work with any of them.

I am wondering if there is anyone who has a working etcd setup with peer to peer certificate validation. I cannot imagine I am the only one running into this problem.

[yichengq]

@ruphin i think you should give ca.crt instead of ca.key for ca file.

[ruphin]

Brilliant. I am an idiot. :)

I don't know what it was but somehow the instructions made it seem like I had to give it the key file. I'll read again and see if I find why.