Skip to content
This repository has been archived by the owner on Sep 30, 2020. It is now read-only.

Generated etcd.pem not following expiration configured in cluster.yaml #892

Closed
iherbmatt opened this issue Aug 30, 2017 · 8 comments
Closed
Labels
kind/bug Categorizes issue or PR as related to a bug.
Milestone

Comments

@iherbmatt
Copy link

Hello,

I recently downloaded kube-aws 0.9.8 and generated a fresh set of credentials using the following command:

kube-aws render credentials

Beforehand, I set both Cert and CA certificate expiration to 3650 days (10 years). What I notice was that all certificates but the etcd.pem cert were generated to expire in 2027. The etcd.pem certificate was set to expire in 365 days.

In order to determine the expiration I'm using the following:
openssl x509 -enddate -noout -in etcd.pem
Here's the output:
notAfter=Aug 30 05:40:25 2018 GMT

It seems this is a bug? It's the only certificate that expires this soon. What would be the result of this certificate expiring?

Thank you,

Matt

@mumoshu
Copy link
Contributor

mumoshu commented Aug 30, 2017

@iherbmatt Good catch!

Probably it is a bug. The value seems to have been hard-coded to 365 days for a year.

https://github.com/kubernetes-incubator/kube-aws/blame/a2daf1a95a40a6326f38fb1c244b0d8a6b790289/core/controlplane/config/encrypted_assets.go#L189-L191

According to the commit history, I believe there's no specific reason to hard-code, especially to 365 days. I'm more than ok to "fix" the expiration to be read from tlsCertDurationDays provided via cluster.yaml, similarly to other certs.

Would you be ok with that?

@mumoshu mumoshu added this to the v0.9.9-rc.1 milestone Aug 30, 2017
@mumoshu mumoshu added the kind/bug Categorizes issue or PR as related to a bug. label Aug 30, 2017
mumoshu added a commit to mumoshu/kube-aws that referenced this issue Aug 30, 2017
@iherbmatt
Copy link
Author

iherbmatt commented Aug 30, 2017 via email

@mumoshu
Copy link
Contributor

mumoshu commented Aug 30, 2017

@iherbmatt Thanks for the confirmation 👍
The fix is on-going in #895

camilb added a commit to camilb/kube-aws that referenced this issue Oct 9, 2017
…-aws

* 'master' of https://github.com/kubernetes-incubator/kube-aws: (55 commits)
  Bump default k8s to 1.7.5
  Fix the scheduling and permissions issue of CA by fixing the conditional in stack-template.json for worker and controller stacks, while making configuration easier.
  Fix tests
  Read worker-ca-key.pem instead of ca-key.pem when certs are managed by kube-aws and TLS bootstrapping is enabled. Formerly, ca-key.pem was read to be install on controller nodes to support TLS boostrapping. However, we have the CA dedicated for TLS boootstrapping today, instead of the one used more widely.
  Fix the bug that symlinks under `credentials` were not accessible from anywhere other than the parent of the `credentials` dir
  Fix the bug that the contents of ca-key.pem was that of ca.pem
  Add more informational log messages to the render-credentials command
  Fix a degradation in message ordering As explained in kubernetes-retired#877 (comment)
  Fix the existing CA support It seems to have broken at some point.
  Fix the json parsing error when clusterAutoscalerSupport is enabled on node pools
  Fix too permissive IAM policy for CA associated to controller nodes Fix kubernetes-retired#903
  Add spot fleet support for the `awsNodeLabels` feature Resolves kubernetes-retired#803
  Drop the ability to disable RBAC via cluster.yaml
  Enable RBAC by default Resolves kubernetes-retired#655
  Fix the hard-coded duration until an etcd cert generated by kube-aws expires Fix kubernetes-retired#892
  Fix test timeouts in CI Resolves kubernetes-retired#893
  Fix ca-key.pem handling and tests
  Create symlinks in test helpers
  Dedicated worker CA and Etcd trusted CA bundle
  Fix encrypted files regen tests
  ...
@whereisaaron
Copy link
Contributor

Bugger I wish I had seen your issue earlier @iherbmatt! I just had a cluster created with an older that 0.9.8 version of kube-aws go haywire a couple hours ago due the etcd cert expiring after one year instead of three like I had specified in cluster.yaml. I had to quickly roll and install a manual replacement certificate! Thanks for fixing this @mumoshu. I checked my 0.9.9 clusters and don't see the problem there.

@iherbmatt
Copy link
Author

iherbmatt commented Jan 8, 2018 via email

@whereisaaron
Copy link
Contributor

@iherbmatt I issued a new certificate, manually installed it in '/etc/etd2/ssl', and restarted etcd with systemctl.

That is not perfect because I imagine a kube-aws update that rolled etcd nodes will install the old certificate again. But I think in this old kube-aws version etcd node are masked from and kube-aws update changes.

@iherbmatt
Copy link
Author

iherbmatt commented Jan 8, 2018 via email

@whereisaaron
Copy link
Contributor

@iherbmatt yeah that occurred to me, but only for one year. I'd have to try using a new kube-aws version to generate it. As unlikely as it sounds, openssl was easier 😄

create-new-etcd-cert.sh

#!/bin/bash
# Generate a new etcd certificate using same CA
set -e

# Must be run in credentials folder
create_cert()
{
  local u="$1" subject="$2" days="$3"

  if [[ -z "$days" ]]; then
    days=1095
  fi
 
  if [[ ! -f ${u}.pem ]]; then
    echo "Creating ${u}.pem for $subject ($days days)"
    openssl genrsa -out ${u}-key.pem 2048
    openssl req -new -key ${u}-key.pem -out ${u}.csr -subj "$subject" -config etcd-new-openssl-config.cnf
    openssl x509 -req -in ${u}.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out ${u}.pem -days $days -extensions v3_req -extfile etcd-new-openssl-config.cnf
    rm ${u}.csr
  else
    echo "${u}.pem already exists"
  fi
}

umask 0077

create_cert etcd-new "/O=kube-aws/CN=kube-etcd"

etcd-new-openssl-config.cnf

[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alternate_names
[ alternate_names ]
DNS.1 = *.ap-southeast-2.compute.internal
DNS.2 = *.ec2.internal

kylehodgetts pushed a commit to HotelsDotCom/kube-aws that referenced this issue Mar 27, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

3 participants