kubeadm: introduce documentation changes for super-admin.conf

- Update most pages where the kubeadm generated admin.conf
is discussed. Include information about the new file "super-admin.conf".

content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_certs_renew_super-admin.conf.md

@@ -0,0 +1,92 @@
+<!--
+The file is auto-generated from the Go source code of the component using a generic
+[generator](https://github.com/kubernetes-sigs/reference-docs/). To learn how
+to generate the reference documentation, please read
+[Contributing to the reference documentation](/docs/contribute/generate-ref-docs/).
+To update the reference content, please follow the
+[Contributing upstream](/docs/contribute/generate-ref-docs/contribute-upstream/)
+guide. You can file document formatting bugs against the
+[reference-docs](https://github.com/kubernetes-sigs/reference-docs/) project.
+-->
+
+
+Renew the certificate embedded in the kubeconfig file for the super-admin
+
+### Synopsis
+
+
+Renew the certificate embedded in the kubeconfig file for the super-admin.
+
+Renewals run unconditionally, regardless of certificate expiration date; extra attributes such as SANs will be based on the existing file/certificates, there is no need to resupply them.
+
+Renewal by default tries to use the certificate authority in the local PKI managed by kubeadm; as alternative it is possible to use K8s certificate API for certificate renewal, or as a last option, to generate a CSR request.
+
+After renewal, in order to make changes effective, is required to restart control-plane components and eventually re-distribute the renewed certificate in case the file is used elsewhere.
+
+```
+kubeadm certs renew super-admin.conf [flags]
+```
+
+### Options
+
+   <table style="width: 100%; table-layout: fixed;">
+<colgroup>
+<col span="1" style="width: 10px;" />
+<col span="1" />
+</colgroup>
+<tbody>
+
+<tr>
+<td colspan="2">--cert-dir string&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: "/etc/kubernetes/pki"</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The path where to save the certificates</p></td>
+</tr>
+
+<tr>
+<td colspan="2">--config string</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Path to a kubeadm configuration file.</p></td>
+</tr>
+
+<tr>
+<td colspan="2">-h, --help</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>help for admin.conf</p></td>
+</tr>
+
+<tr>
+<td colspan="2">--kubeconfig string&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: "/etc/kubernetes/admin.conf"</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.</p></td>
+</tr>
+
+</tbody>
+</table>
+
+
+
+### Options inherited from parent commands
+
+   <table style="width: 100%; table-layout: fixed;">
+<colgroup>
+<col span="1" style="width: 10px;" />
+<col span="1" />
+</colgroup>
+<tbody>
+
+<tr>
+<td colspan="2">--rootfs string</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>[EXPERIMENTAL] The path to the 'real' host root filesystem.</p></td>
+</tr>
+
+</tbody>
+</table>
+
+
+

content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_kubeconfig_super-admin.md

@@ -0,0 +1,121 @@
+<!--
+The file is auto-generated from the Go source code of the component using a generic
+[generator](https://github.com/kubernetes-sigs/reference-docs/). To learn how
+to generate the reference documentation, please read
+[Contributing to the reference documentation](/docs/contribute/generate-ref-docs/).
+To update the reference content, please follow the
+[Contributing upstream](/docs/contribute/generate-ref-docs/contribute-upstream/)
+guide. You can file document formatting bugs against the
+[reference-docs](https://github.com/kubernetes-sigs/reference-docs/) project.
+-->
+
+
+Generate a kubeconfig file for the super-admin
+
+### Synopsis
+
+
+Generate a kubeconfig file for the super-admin.
+
+```
+kubeadm init phase kubeconfig super-admin [flags]
+```
+
+### Options
+
+   <table style="width: 100%; table-layout: fixed;">
+<colgroup>
+<col span="1" style="width: 10px;" />
+<col span="1" />
+</colgroup>
+<tbody>
+
+<tr>
+<td colspan="2">--apiserver-advertise-address string</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The IP address the API Server will advertise it's listening on. If not set the default network interface will be used.</p></td>
+</tr>
+
+<tr>
+<td colspan="2">--apiserver-bind-port int32&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: 6443</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Port for the API Server to bind to.</p></td>
+</tr>
+
+<tr>
+<td colspan="2">--cert-dir string&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: "/etc/kubernetes/pki"</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The path where to save and store the certificates.</p></td>
+</tr>
+
+<tr>
+<td colspan="2">--config string</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Path to a kubeadm configuration file.</p></td>
+</tr>
+
+<tr>
+<td colspan="2">--control-plane-endpoint string</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Specify a stable IP address or DNS name for the control plane.</p></td>
+</tr>
+
+<tr>
+<td colspan="2">--dry-run</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Don't apply any changes; just output what would be done.</p></td>
+</tr>
+
+<tr>
+<td colspan="2">-h, --help</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>help for admin</p></td>
+</tr>
+
+<tr>
+<td colspan="2">--kubeconfig-dir string&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: "/etc/kubernetes"</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The path where to save the kubeconfig file.</p></td>
+</tr>
+
+<tr>
+<td colspan="2">--kubernetes-version string&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: "stable-1"</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Choose a specific Kubernetes version for the control plane.</p></td>
+</tr>
+
+</tbody>
+</table>
+
+
+
+### Options inherited from parent commands
+
+   <table style="width: 100%; table-layout: fixed;">
+<colgroup>
+<col span="1" style="width: 10px;" />
+<col span="1" />
+</colgroup>
+<tbody>
+
+<tr>
+<td colspan="2">--rootfs string</td>
+</tr>
+<tr>
+<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>[EXPERIMENTAL] The path to the 'real' host root filesystem.</p></td>
+</tr>
+
+</tbody>
+</table>
+
+
+

content/en/docs/reference/setup-tools/kubeadm/implementation-details.md

@@ -64,6 +64,7 @@ in a majority of cases, and the most intuitive location; other constants paths a
   - `controller-manager.conf`
   - `scheduler.conf`
   - `admin.conf` for the cluster admin and kubeadm itself
+  - `super-admin.conf` for the cluster super-admin that can bypass RBAC
 
 - Names of certificates and key files :
 
@@ -209,12 +210,21 @@ Kubeadm generates kubeconfig files with identities for control plane components:
   This client cert should have the CN `system:kube-scheduler`, as defined by default
   [RBAC core components roles](/docs/reference/access-authn-authz/rbac/#core-component-roles)
 
-Additionally, a kubeconfig file for kubeadm itself and the admin is generated and saved into the
-`/etc/kubernetes/admin.conf` file.  The "admin" here is defined as the actual person(s) that is
-administering the cluster and wants to have full control (**root**) over the cluster.  The
-embedded client certificate for admin should be in the `system:masters` organization, as defined
-by default [RBAC user facing role bindings](/docs/reference/access-authn-authz/rbac/#user-facing-roles).
-It should also include a CN. Kubeadm uses the `kubernetes-admin` CN.
+Additionally, a kubeconfig file for kubeadm as an administrative entity is generated and stored
+in `/etc/kubernetes/admin.conf`. This file includes a certificate with
+`Subject: O = kubeadm:cluster-admins, CN = kubernetes-admin`. `kubeadm:cluster-admins`
+is a group managed by kubeadm. It is bound to the `cluster-admin` ClusterRole during `kubeadm init`,
+by using the `super-admin.conf` file, which does not require RBAC.
+This `admin.conf` file must remain on control plane nodes and not be shared with additional users.
+
+During `kubeadm init` another kubeconfig file is generated and stored in `/etc/kubernetes/super-admin.conf`.
+This file includes a certificate with `Subject: O = system:masters, CN = kubernetes-super-admin`.
+`system:masters` is a super user group that bypasses RBAC and makes `super-admin.conf` useful in case
+of an emergency where a cluster is locked due to RBAC misconfiguration.
+The `super-admin.conf` file can be stored in a safe location and not shared with additional users.
+
+See [RBAC user facing role bindings](/docs/reference/access-authn-authz/rbac/#user-facing-roles)
+for additional information RBAC and built-in ClusterRoles and groups.
 
 Please note that:
 

content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md

@@ -34,6 +34,7 @@ For more details see [Manual certificate renewal](/docs/tasks/administer-cluster
 {{< tab name="etcd-server" include="generated/kubeadm_certs_renew_etcd-server.md" />}}
 {{< tab name="front-proxy-client" include="generated/kubeadm_certs_renew_front-proxy-client.md" />}}
 {{< tab name="scheduler.conf" include="generated/kubeadm_certs_renew_scheduler.conf.md" />}}
+{{< tab name="super-admin.conf" include="generated/kubeadm_certs_renew_super-admin.conf.md" />}}
 {{< /tabs >}}
 
 ## kubeadm certs certificate-key {#cmd-certs-certificate-key}

content/en/docs/reference/setup-tools/kubeadm/kubeadm-init-phase.md

@@ -58,6 +58,7 @@ You can create all required kubeconfig files by calling the `all` subcommand or
 {{< tab name="kubelet" include="generated/kubeadm_init_phase_kubeconfig_kubelet.md" />}}
 {{< tab name="controller-manager" include="generated/kubeadm_init_phase_kubeconfig_controller-manager.md" />}}
 {{< tab name="scheduler" include="generated/kubeadm_init_phase_kubeconfig_scheduler.md" />}}
+{{< tab name="super-admin" include="generated/kubeadm_init_phase_kubeconfig_super-admin.md" />}}
 {{< /tabs >}}
 
 ## kubeadm init phase control-plane {#cmd-phase-control-plane}

content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md

@@ -32,8 +32,9 @@ following steps:
    arguments, lowercased if necessary.
 
 1. Writes kubeconfig files in `/etc/kubernetes/` for the kubelet, the controller-manager and the
-   scheduler to use to connect to the API server, each with its own identity, as well as an
-   additional kubeconfig file for administration named `admin.conf`.
+   scheduler to use to connect to the API server, each with its own identity. Also
+   additional kubeconfig files are written, for kubeadm as administrative entity (`admin.conf`)
+   and for a super admin user that can bypass RBAC (`super-admin.conf`).
 
 1. Generates static Pod manifests for the API server,
    controller-manager and scheduler. In case an external etcd is not provided,
@@ -186,7 +187,7 @@ as a learner and promoted to a voting member only after the etcd data are fully
 List of deprecated feature gates:
 
 {{< table caption="kubeadm deprecated feature gates" >}}
-Feature | Default 
+Feature | Default
 :-------|:--------
 `UpgradeAddonsBeforeControlPlane` | `false`
 {{< /table >}}
@@ -291,7 +292,7 @@ for etcd and CoreDNS.
 
 #### Custom sandbox (pause) images {#custom-pause-image}
 
-To set a custom image for these you need to configure this in your 
+To set a custom image for these you need to configure this in your
 {{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}
 to use the image.
 Consult the documentation for your container runtime to find out how to change this setting;
@@ -386,8 +387,9 @@ DNS name or an address of a load balancer.
    kubeadm certs certificate-key
    ```
 
-Once the cluster is up, you can grab the admin credentials from the control-plane node
-at `/etc/kubernetes/admin.conf` and use that to talk to the cluster.
+Once the cluster is up, you can use the `/etc/kubernetes/admin.conf` file from
+a control-plane node to talk to the cluster with administrator credentials or
+[Generating kubeconfig files for additional users](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#kubeconfig-additional-users).
 
 Note that this style of bootstrap has some relaxed security guarantees because
 it does not allow the root CA hash to be validated with

content/en/docs/setup/best-practices/certificates.md

@@ -184,19 +184,36 @@ you need to provide if you are generating all of your own keys and certificates:
 
 You must manually configure these administrator account and service accounts:
 
-| filename                | credential name            | Default CN                          | O (in Subject) |
-|-------------------------|----------------------------|-------------------------------------|----------------|
-| admin.conf              | default-admin              | kubernetes-admin                    | system:masters |
-| kubelet.conf            | default-auth               | system:node:`<nodeName>` (see note) | system:nodes   |
-| controller-manager.conf | default-controller-manager | system:kube-controller-manager      |                |
-| scheduler.conf          | default-scheduler          | system:kube-scheduler               |                |
+| filename                | credential name            | Default CN                          | O (in Subject)         |
+|-------------------------|----------------------------|-------------------------------------|------------------------|
+| admin.conf              | default-admin              | kubernetes-admin                    | `<admin-group>`        |
+| super-admin.conf        | default-super-admin        | kubernetes-super-admin              | system:masters         |
+| kubelet.conf            | default-auth               | system:node:`<nodeName>` (see note) | system:nodes           |
+| controller-manager.conf | default-controller-manager | system:kube-controller-manager      |                        |
+| scheduler.conf          | default-scheduler          | system:kube-scheduler               |                        |
 
 {{< note >}}
 The value of `<nodeName>` for `kubelet.conf` **must** match precisely the value of the node name
 provided by the kubelet as it registers with the apiserver. For further details, read the
 [Node Authorization](/docs/reference/access-authn-authz/node/).
 {{< /note >}}
 
+{{< note >}}
+In the above example `<admin-group>` is implementation specific. Some tools sign the
+certificate in the default `admin.conf` to be part of the `system:masters` group.
+`system:masters` is a break-glass, super user group can bypass the authorization
+layer of Kubernetes, such as RBAC. Also some tools do not generate a separate
+`super-admin.conf` with a certificate bound to this super user group.
+
+kubeadm generates two separate administrator certificates in kubeconfig files.
+One is in `admin.conf` and has `Subject: O = kubeadm:cluster-admins, CN = kubernetes-admin`.
+`kubeadm:cluster-admins` is a custom group bound to the `cluster-admin` ClusterRole.
+This file is generated on all kubeadm managed control plane machines.
+
+Another is in `super-admin.conf` that has `Subject: O = system:masters, CN = kubernetes-super-admin`.
+This file is generated only on the node where `kubeadm init` was called.
+{{< /note >}}
+
 1. For each config, generate an x509 cert/key pair with the given CN and O.
 
 1. Run `kubectl` as follows for each config:
@@ -213,6 +230,7 @@ These files are used as follows:
 | filename                | command                 | comment                                                               |
 |-------------------------|-------------------------|-----------------------------------------------------------------------|
 | admin.conf              | kubectl                 | Configures administrator user for the cluster                         |
+| super-admin.conf        | kubectl                 | Configures super administrator user for the cluster                   |
 | kubelet.conf            | kubelet                 | One required for each node in the cluster.                            |
 | controller-manager.conf | kube-controller-manager | Must be added to manifest in `manifests/kube-controller-manager.yaml` |
 | scheduler.conf          | kube-scheduler          | Must be added to manifest in `manifests/kube-scheduler.yaml`          |
@@ -221,6 +239,7 @@ The following files illustrate full paths to the files listed in the previous ta
 
 ```
 /etc/kubernetes/admin.conf
+/etc/kubernetes/super-admin.conf
 /etc/kubernetes/kubelet.conf
 /etc/kubernetes/controller-manager.conf
 /etc/kubernetes/scheduler.conf

content/en/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md

@@ -211,11 +211,19 @@ export KUBECONFIG=/etc/kubernetes/admin.conf
 ```
 
 {{< warning >}}
-Kubeadm signs the certificate in the `admin.conf` to have `Subject: O = system:masters, CN = kubernetes-admin`.
-`system:masters` is a break-glass, super user group that bypasses the authorization layer (e.g. RBAC).
-Do not share the `admin.conf` file with anyone and instead grant users custom permissions by generating
-them a kubeconfig file using the `kubeadm kubeconfig user` command. For more details see
-[Generating kubeconfig files for additional users](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#kubeconfig-additional-users).
+The kubeconfig file `admin.conf` that `kubeadm init` generates contains a certificate with
+`Subject: O = kubeadm:cluster-admins, CN = kubernetes-admin`. The group `kubeadm:cluster-admins`
+is bound to the built-in `cluster-admin` ClusterRole.
+Do not share the `admin.conf` file with anyone.
+
+`kubeadm init` generates another kubeconfig file `super-admin.conf` that contains a certificate with
+`Subject: O = system:masters, CN = kubernetes-super-admin`.
+`system:masters` is a break-glass, super user group that bypasses the authorization layer (for example RBAC).
+Do not share the `super-admin.conf` file with anyone. It is recommended to move the file to a safe location.
+
+See
+[Generating kubeconfig files for additional users](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#kubeconfig-additional-users)
+on how to use `kubeadm kubeconfig user` to generate kubeconfig files for additional users.
 {{< /warning >}}
 
 Make a record of the `kubeadm join` command that `kubeadm init` outputs. You