Dedicated worker CA and Etcd trusted CA bundle

[redbaron]

TLSBootstap CA is now loaded from worker-ca.pem
Etcd trusted CA bundle is now loaded from etcd-trusted-ca.pem

Both files by default are symlinks to a well known ca.pem

This change enables advanced, more secure setup, where certificates
stored on worker nodes are trusted only by APIserver and NOT
Etcd server. This kind of setup is not yet supported by
"kube-aws render credentials" command out of the box, and
therefore currently requires users to be generating and
encrypting certs on their own to benefit from it.

Depends on #882

[codecov-io]

Codecov Report

Merging #885 into master will increase coverage by 0.64%.
The diff coverage is 63.79%.

@@            Coverage Diff             @@
##           master     #885      +/-   ##
==========================================
+ Coverage   34.77%   35.41%   +0.64%     
==========================================
  Files          57       57              
  Lines        3919     3978      +59     
==========================================
+ Hits         1363     1409      +46     
- Misses       2396     2406      +10     
- Partials      160      163       +3

Impacted Files	Coverage Δ
core/controlplane/config/credential.go	53.98% <37.93%> (+2.55%)	⬆️
core/controlplane/config/encrypted_assets.go	70.41% <72.41%> (+0.75%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 136f850...8bcad3b. Read the comment docs.

[mumoshu]

@redbaron Thanks for your efforts. I like the motivation behind this change 👍

/lgtm

[mumoshu]

r.CACert here should be r.CAKey

[mumoshu]

to should be just sl.to while calling chdir before/after creating the symlink. Otherwise we get mysterious error when tried to read it from somewhere other than the parent directory of credentials/:

# The file is there
$ ls assets/k8s88/credentials/worker-ca-key.pem
assets/k8s88/credentials/worker-ca-key.pem

# However, actually trying to read it results in a no-such-file error
$ cat assets/k8s88/credentials/worker-ca-key.pem
cat: assets/k8s88/credentials/worker-ca-key.pem: No such file or directory

$ ls -lah
total 136
drwxr-xr-x  20 mumoshu  staff   680B  9  1 15:29 .
drwxr-xr-x   7 mumoshu  staff   238B  9  1 15:29 ..
-rw-r--r--   1 mumoshu  staff     1B  9  1 15:29 .gitignore
-rw-------   1 mumoshu  staff   1.6K  9  1 15:29 admin-key.pem
-rw-------   1 mumoshu  staff   1.1K  9  1 15:29 admin.pem
-rw-------   1 mumoshu  staff   1.6K  9  1 15:29 apiserver-key.pem
-rw-------   1 mumoshu  staff   1.2K  9  1 15:29 apiserver.pem
-rw-------   1 mumoshu  staff   1.6K  9  1 15:29 ca-key.pem
-rw-------   1 mumoshu  staff   1.0K  9  1 15:29 ca.pem
-rw-------   1 mumoshu  staff   1.6K  9  1 15:29 etcd-client-key.pem
-rw-------   1 mumoshu  staff   1.1K  9  1 15:29 etcd-client.pem
-rw-------   1 mumoshu  staff   1.6K  9  1 15:29 etcd-key.pem
lrwxr-xr-x   1 mumoshu  staff    18B  9  1 15:29 etcd-trusted-ca.pem -> credentials/ca.pem
-rw-------   1 mumoshu  staff   1.1K  9  1 15:29 etcd.pem
-rw-------   1 mumoshu  staff   344B  9  1 15:29 kubelet-tls-bootstrap-token
-rw-------   1 mumoshu  staff     0B  9  1 15:29 tokens.csv
lrwxr-xr-x   1 mumoshu  staff    22B  9  1 15:29 worker-ca-key.pem -> credentials/ca-key.pem
lrwxr-xr-x   1 mumoshu  staff    18B  9  1 15:29 worker-ca.pem -> credentials/ca.pem
-rw-------   1 mumoshu  staff   1.6K  9  1 15:29 worker-key.pem
-rw-------   1 mumoshu  staff   1.1K  9  1 15:29 worker.pem

For example, etcd-trusted-ca.pem -> credentials/ca.pem should be etcd-trusted-ca.pem -> ca.pem

[mumoshu]

Here's my current work-around:

--- a/core/controlplane/config/encrypted_assets.go
+++ b/core/controlplane/config/encrypted_assets.go
@@ -509,8 +509,18 @@ func (r *RawAssetsOnMemory) WriteToDir(dirname string, includeCAKey bool) error
                {"ca-key.pem", "worker-ca-key.pem"},
        }

+       wd, err := os.Getwd()
+       if err != nil {
+               return err
+       }
+
+       if err := os.Chdir(dirname); err != nil {
+               return err
+       }
+
        for _, sl := range symlinks {
-               to := filepath.Join(dirname, sl.to)
+               from := sl.from
+               to := sl.to

                if _, err := os.Lstat(to); err == nil {
                        if err := os.Remove(to); err != nil {
@@ -518,10 +528,15 @@ func (r *RawAssetsOnMemory) WriteToDir(dirname string, includeCAKey bool) error
                        }
                }

-               if err := os.Symlink(sl.from, to); err != nil {
+               if err := os.Symlink(from, to); err != nil {
                        return err
                }
        }
+
+       if err := os.Chdir(wd); err != nil {
+               return err
+       }

[mumoshu]

worker-ca-key.pem becomes a symlink to an inexistent file when ca-key.pem was not written because the user provided --ca-key-path while the TLS bootstrapping is disabled.
We should not create the worker-ca-key.pem symlink or try to read it afterwards when includeCAKey is false, the same as what we do for ca-key.pem in #877