modules/*: trust CA certificates on the nodes #2362

s-urbaniak · 2017-11-10T16:26:40Z

This also bumps the ignition provider to 1.0.0 to support S3 downloads
via ignition.

Fixes #INST-67

coreosbot · 2017-11-10T16:26:41Z

Can one of the admins verify this patch?

s-urbaniak · 2017-11-14T15:59:30Z

@squat PTAL

squat

Looks great overall. I found one issue that I think breaks GCP. PTAL

squat · 2017-11-15T14:10:52Z

modules/gcp/worker-igm/ignition.tf

@@ -3,6 +3,7 @@ data "ignition_config" "main" {
    "${data.ignition_file.kubeconfig.id}",
    "${var.ign_max_user_watches_id}",
    "${var.ign_installer_kubelet_env_id}",
+    "${var.ign_ca_cert_id_list}",
  ]

  systemd = [


The worker is missing the var.ign_update_ca_certificates_dropin_id service

fixed, thanks!

squat · 2017-11-15T14:18:45Z

platforms/metal/cl/bootkube-controller.yaml.tmpl

@@ -38,6 +38,11 @@ systemd:
    - name: bootkube.service
      enable: false
      contents: {{.ign_bootkube_service_json}}
+    - name: update-ca-certificates.service


not for this pr but we really need to refactor this platform so that it uses the same ignition format as all the other platforms

totally agreed 👍

squat · 2017-11-17T15:45:01Z

ok to test

s-urbaniak · 2017-11-17T16:19:58Z

This builds on top of #2421

s-urbaniak · 2017-11-17T16:26:52Z

/cc @sym3tri @squat

this will need to slip unfortunately :-( despite the countermeasures in #2422 this exceeds the userdata limit on AWS. We need to move non-ciritical (security wise) assets to S3 to free up space in userdata unfortunately. This implies a larger refactoring though.

This also bumps the ignition provider to 1.0.0 to support S3 downloads via ignition. Fixes #INST-67

Currently etcd certificate ignition files are created in their respective platform module. This unifies it by declaring the ignition file units centrally in the ignition module.

Currently the etcd TLS module also generates a zip file which is only used on AWS to reduce the userdata size to be <20k (hard limit on AWS). Since the etcd TLS assets will be provisioned via S3 this optimization/hack is not needed any more.

We hit the limits of the AWS userdata limit (20k) constantly. This fixes it by provisioning a minimal ignition configuration only which points to a replacement ignition configuration hosted on S3. This also removes workarounds/hacks to keep the userdata size small, especially for provisioning TLS assets on etcd nodes.

s-urbaniak · 2017-11-21T11:41:15Z

@squat @enxebre PTAL

s-urbaniak · 2017-11-21T15:51:34Z

retest this please

squat

LGTM once green

s-urbaniak · 2017-11-21T16:21:39Z

modules/ignition/resources/dropins/10-always-update-ca-certificates.conf

+
+[Service]
+ExecStart=
+ExecStart=/usr/sbin/update-ca-certificates


/cc @lucab This works, because we are enforcing rehashing all files. Rather the default service should work, but openssl is pointing to the wrong default bundle.

s-urbaniak · 2017-11-22T14:51:46Z

all green, merging

sym3tri · 2017-12-11T17:08:09Z

modules/aws/etcd/nodes.tf

+      "Action": [
+        "ecr:DescribeRepositories",
+        "ecr:ListImages",
+        "ecr:BatchGetImage"


@s-urbaniak Why did you need to add ECR permissions?

It might've been a copy&pasta oversight, but without this the machine didn't seem to boot.

* modules/*: trust CA certificates on the nodes This also bumps the ignition provider to 1.0.0 to support S3 downloads via ignition. Fixes #INST-67 * modules/ignition: unify etcd certificate generation Currently etcd certificate ignition files are created in their respective platform module. This unifies it by declaring the ignition file units centrally in the ignition module. * modules/tls/etcd: remove zip generation Currently the etcd TLS module also generates a zip file which is only used on AWS to reduce the userdata size to be <20k (hard limit on AWS). Since the etcd TLS assets will be provisioned via S3 this optimization/hack is not needed any more. * */aws: use S3 for ignition provisioning We hit the limits of the AWS userdata limit (20k) constantly. This fixes it by provisioning a minimal ignition configuration only which points to a replacement ignition configuration hosted on S3. This also removes workarounds/hacks to keep the userdata size small, especially for provisioning TLS assets on etcd nodes. * Documentation/examples: regenerate * */azure: use unified etcd TLS ignition files * */gcp: use unified etcd TLS ignition files * */openstack: use unified etcd TLS ignition files * */vmware: use unified etcd TLS ignition files

wking · 2018-07-23T20:40:22Z

modules/aws/etcd/ignition_s3.tf

@@ -0,0 +1,25 @@
+resource "aws_s3_bucket_object" "ignition_etcd" {
+  count = "${length(var.external_endpoints) == 0 ? var.instance_count : 0}"


I'm new to Terraform and was scratching my head about this after not finding count in the aws_s3_bucket_object docs. It turns out it is a meta-parameter (more here). I thought I'd add this note in case it helps any other Terraform newbies ;).

The local variable is from cd71f30 (Ignition assets, 2018-02-14, coreos/tectonic-installer#2940), but it doesn't scale beyond three etcd nodes. This commit pivots to an approach that scales to an arbitrary number of nodes, matching the scalable generation logic we've had since 0c7ac90 (modules/*: trust CA certificates on the nodes, 2017-11-22, coreos/tectonic-installer#2362).

s-urbaniak added do-not-merge platform/aws platform/azure platform/bare-metal run-smoke-tests labels Nov 10, 2017

s-urbaniak force-pushed the inst-67-new branch from 2740e4e to 552ecfd Compare November 14, 2017 15:56

s-urbaniak changed the title ~~[WIP] modules/*: trust CA certificates on the nodes~~ modules/*: trust CA certificates on the nodes Nov 14, 2017

s-urbaniak removed the do-not-merge label Nov 14, 2017

squat suggested changes Nov 15, 2017

View reviewed changes

s-urbaniak force-pushed the inst-67-new branch from 552ecfd to c82dfb9 Compare November 17, 2017 14:23

s-urbaniak force-pushed the inst-67-new branch from c82dfb9 to bcffef6 Compare November 17, 2017 16:19

s-urbaniak added the do-not-merge label Nov 17, 2017

s-urbaniak changed the title ~~modules/*: trust CA certificates on the nodes~~ [WIP] modules/*: trust CA certificates on the nodes Nov 17, 2017

Sergiusz Urbaniak added 9 commits November 21, 2017 12:20

modules/*: trust CA certificates on the nodes

447af6d

This also bumps the ignition provider to 1.0.0 to support S3 downloads via ignition. Fixes #INST-67

modules/ignition: unify etcd certificate generation

c8e828a

Currently etcd certificate ignition files are created in their respective platform module. This unifies it by declaring the ignition file units centrally in the ignition module.

modules/tls/etcd: remove zip generation

9c133ab

Currently the etcd TLS module also generates a zip file which is only used on AWS to reduce the userdata size to be <20k (hard limit on AWS). Since the etcd TLS assets will be provisioned via S3 this optimization/hack is not needed any more.

*/azure: use unified etcd TLS ignition files

046c277

*/gcp: use unified etcd TLS ignition files

532ff5b

*/openstack: use unified etcd TLS ignition files

f328eb9

*/vmware: use unified etcd TLS ignition files

dd2e0fd

Documentation/examples: regenerate

6d66b86

s-urbaniak force-pushed the inst-67-new branch from bcffef6 to 6d66b86 Compare November 21, 2017 11:36

s-urbaniak changed the title ~~[WIP] modules/*: trust CA certificates on the nodes~~ modules/*: trust CA certificates on the nodes Nov 21, 2017

s-urbaniak removed the do-not-merge label Nov 21, 2017

s-urbaniak added the platform/gcp label Nov 21, 2017

squat approved these changes Nov 21, 2017

View reviewed changes

s-urbaniak commented Nov 21, 2017

View reviewed changes

s-urbaniak merged commit 0c7ac90 into coreos:master Nov 22, 2017

lblackstone mentioned this pull request Nov 27, 2017

Trust the Kubernetes CA on nodes #1351

Closed

etoews mentioned this pull request Nov 28, 2017

Install kube-ca on all nodes #736

Closed

sym3tri reviewed Dec 11, 2017

View reviewed changes

cpanato mentioned this pull request Dec 11, 2017

docs: add missing permission coreos/tectonic-docs#105

Merged

enxebre mentioned this pull request Dec 19, 2017

Add govcloud support #2594

Closed

wking reviewed Jul 23, 2018

View reviewed changes

wking mentioned this pull request Jul 23, 2018

modules/aws/etcd/ignition: Drop hard-coded ignition_etcd_keys openshift/installer#65

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

modules/*: trust CA certificates on the nodes #2362

modules/*: trust CA certificates on the nodes #2362

s-urbaniak commented Nov 10, 2017

coreosbot commented Nov 10, 2017

s-urbaniak commented Nov 14, 2017

squat left a comment

squat Nov 15, 2017

s-urbaniak Nov 17, 2017

squat Nov 15, 2017

s-urbaniak Nov 17, 2017

squat commented Nov 17, 2017

s-urbaniak commented Nov 17, 2017

s-urbaniak commented Nov 17, 2017

s-urbaniak commented Nov 21, 2017

s-urbaniak commented Nov 21, 2017

squat left a comment

s-urbaniak Nov 21, 2017 •

edited

Loading

s-urbaniak commented Nov 22, 2017

sym3tri Dec 11, 2017

s-urbaniak Dec 12, 2017

wking Jul 23, 2018

		@@ -0,0 +1,25 @@
		resource "aws_s3_bucket_object" "ignition_etcd" {
		count = "${length(var.external_endpoints) == 0 ? var.instance_count : 0}"

modules/*: trust CA certificates on the nodes #2362

modules/*: trust CA certificates on the nodes #2362

Conversation

s-urbaniak commented Nov 10, 2017

coreosbot commented Nov 10, 2017

s-urbaniak commented Nov 14, 2017

squat left a comment

Choose a reason for hiding this comment

squat Nov 15, 2017

Choose a reason for hiding this comment

s-urbaniak Nov 17, 2017

Choose a reason for hiding this comment

squat Nov 15, 2017

Choose a reason for hiding this comment

s-urbaniak Nov 17, 2017

Choose a reason for hiding this comment

squat commented Nov 17, 2017

s-urbaniak commented Nov 17, 2017

s-urbaniak commented Nov 17, 2017

s-urbaniak commented Nov 21, 2017

s-urbaniak commented Nov 21, 2017

squat left a comment

Choose a reason for hiding this comment

s-urbaniak Nov 21, 2017 • edited Loading

Choose a reason for hiding this comment

s-urbaniak commented Nov 22, 2017

sym3tri Dec 11, 2017

Choose a reason for hiding this comment

s-urbaniak Dec 12, 2017

Choose a reason for hiding this comment

wking Jul 23, 2018

Choose a reason for hiding this comment

s-urbaniak Nov 21, 2017 •

edited

Loading