From bc986c5708b6d176c2b0b97b0731de34401acedc Mon Sep 17 00:00:00 2001
From: Hongcai Ren <renhongcai@huawei.com>
Date: Sat, 26 Oct 2019 15:51:23 +0800
Subject: [PATCH] Merged 1.14~1.16 changes (#17117)

---
 .../docs/concepts/containers/runtime-class.md | 360 +++++++++++++-----
 1 file changed, 265 insertions(+), 95 deletions(-)

diff --git a/content/zh/docs/concepts/containers/runtime-class.md b/content/zh/docs/concepts/containers/runtime-class.md
index 4029826901f0d..8328a6ecfc674 100644
--- a/content/zh/docs/concepts/containers/runtime-class.md
+++ b/content/zh/docs/concepts/containers/runtime-class.md
@@ -2,170 +2,162 @@
 reviewers:
 - tallclair
 - dchen1107
-title: RuntimeClass
+title: Runtime Class
 content_template: templates/concept
 weight: 20
 ---
 
 {{% capture overview %}}
 
-{{< feature-state for_k8s_version="v1.12" state="alpha" >}}
+{{< feature-state for_k8s_version="v1.14" state="beta" >}}
 
 <!-- 
 This page describes the RuntimeClass resource and runtime selection mechanism.
 -->
-本页面讨论了 RuntimeClass 资源和运行时的选择机制。
+本页面描述了 RuntimeClass 资源和运行时的选择机制。
+
+{{< warning >}}
+<!--
+RuntimeClass includes *breaking* changes in the beta upgrade in v1.14. If you were using
+RuntimeClass prior to v1.14, see [Upgrading RuntimeClass from Alpha to
+Beta](#upgrading-runtimeclass-from-alpha-to-beta).
+-->RuntimeClass 特性在 v1.14 版本升级为 beta 特性时引入了不兼容的改变。
+如果你在 v1.14 以前的版本中使用 RuntimeClass，请查阅
+[Upgrading RuntimeClass from Alpha to Beta](#upgrading-runtimeclass-from-alpha-to-beta)。
+{{< /warning >}}
 
 {{% /capture %}}
 
-{{< toc >}}
 
 {{% capture body %}}
 
-## RuntimeClass
+## Runtime Class
 
-<!-- 
-RuntimeClass is an alpha feature for selecting the container runtime configuration to use to run a
-pod's containers.
--->
-RuntimeClass 是一个 alpha 功能，用来选择运行 pod 中容器的容器运行时配置。
-
-<!-- 
-### Set Up
+<!--
+RuntimeClass is a feature for selecting the container runtime configuration. The container runtime
+configuration is used to run a Pod's containers.
 -->
-### 设置
+RuntimeClass 是用于选择容器运行时配置的特性，容器运行时配置用于运行 Pod 中的容器。
 
-<!-- 
-As an early alpha feature, there are some additional setup steps that must be taken in order to use
-the RuntimeClass feature:
--->
-作为一个处于早期状态的 alpha 特性，必须要完成以下步骤才能使用 RuntimeClass 功能：
+## Motivation
 
 <!--
-1. Enable the RuntimeClass feature gate (on apiservers & kubelets, requires version 1.12+)
-2. Install the RuntimeClass CRD
-3. Configure the CRI implementation on nodes (runtime dependent)
-4. Create the corresponding RuntimeClass resources
+You can set a different RuntimeClass between different Pods to provide a balance of
+performance versus security. For example, if part of your workload deserves a high
+level of information security assurance, you might choose to schedule those Pods so
+that they run in a container runtime that uses hardware virtualization. You'd then
+benefit from the extra isolation of the alternative runtime, at the expense of some
+additional overhead.
 -->
-1. 开启 RuntimeClass 特性门控（在 apiservers ＆ kubelets 上，需要 1.12 及以上版本）
-2. 安装 RuntimeClass CRD
-3. 在节点上配置 CRI 的实现（取决于所选的运行时）
-4. 创建相应的 RuntimeClass 资源
+您可以在不同的 pod 之间设置不同的 RuntimeClass，以提供性能与安全性之间的平衡。
+例如，如果您的部分工作负载需要高级别的信息安全保证，那么您可以选择性地调度这些 pod，
+使它们在使用硬件虚拟化的容器运行时中运行。
+然后，您将从可选运行时的额外隔离中获益，代价是一些额外的开销。
 
 <!--
-#### 1. Enable the RuntimeClass feature gate
+You can also use RuntimeClass to run different Pods with the same container runtime
+but with different settings.
 -->
-#### 1. 开启 RuntimeClass 特性门控
+您还可以使用 RuntimeClass 运行具有相同容器运行时但具有不同设置的pod。
 
-<!--
-See [Feature Gates](/docs/reference/command-line-tools-reference/feature-gates/) for an explanation
-of enabling feature gates. The RuntimeClass feature gate must be enabled on apiservers _and_ kubelets.
+<!-- 
+### Set Up
 -->
-参见[特性门控](/docs/reference/command-line-tools-reference/feature-gates/)文档了解如何开启特定的特性门控。
-`RuntimeClass` 特性门控必须在 apiservers _和_ kubelets 上同时开启，才能使用。
+### 设置
 
 <!--
-#### 2. Install the RuntimeClass CRD
+Ensure the RuntimeClass feature gate is enabled (it is by default). See [Feature
+Gates](/docs/reference/command-line-tools-reference/feature-gates/) for an explanation of enabling
+feature gates. The `RuntimeClass` feature gate must be enabled on apiservers _and_ kubelets.
 -->
-#### 2. 安装 RuntimeClass CRD
+确保 RuntimeClass 特性开关处于开启状态（默认为开启状态）。
+关于特性开关的详细介绍，请查阅
+[Feature Gates](/docs/reference/command-line-tools-reference/feature-gates/)。
+`RuntimeClass` 特性开关必须在 apiservers 和 kubelets 同时开启。
 
 <!--
-The RuntimeClass [CustomResourceDefinition][/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/] (CRD) can be found in the addons directory of the
-Kubernetes git repo:
+1. Configure the CRI implementation on nodes (runtime dependent)
+2. Create the corresponding RuntimeClass resources
 -->
-可以在 Kubernetes git 仓库的 addons 目录中找到
-RuntimeClass [CustomResourceDefinition (CRD)](/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/) 的定义：
 
-https://github.com/kubernetes/kubernetes/tree/release-1.12/cluster/addons/runtimeclass/runtimeclass_crd.yaml
+1. 在节点上配置 CRI 的实现（取决于所选用的运行时）
+2. 创建相应的 RuntimeClass 资源
 
 <!--
-Install the CRD with `kubectl apply -f runtimeclass_crd.yaml`.
+#### 1. Configure the CRI implementation on nodes
 -->
-
-使用 `kubectl apply -f runtimeclass_crd.yaml` 命令安装 CRD。
+#### 1. 在节点上配置 CRI 实现
 
 <!--
-#### 3. Configure the CRI implementation on nodes
+The configurations available through RuntimeClass are Container Runtime Interface (CRI)
+implementation dependent. See the corresponding documentation ([below](#cri-configuration)) for your
+CRI implementation for how to configure.
 -->
-#### 3. 在节点上配置 CRI 实现
+RuntimeClass 的配置依赖于 运行时接口（CRI）的实现。
+根据你使用的 CRI 实现，查阅相关的文档（[下方](#cri-configuration)）来了解如何配置。
 
+{{< note >}}
 <!--
-The configurations to select between with RuntimeClass are CRI implementation dependent. See the
-corresponding documentation for your CRI implementation for how to configure. As this is an alpha
-feature, not all CRIs support multiple RuntimeClasses yet.
--->
-各 RuntimeClass 所支持的配置选项取决于 CRI 的实现本身。
-请参阅 CRI 实现的相应文档了解具体如何配置。
-由于这是一个 alpha 特性功能，并非所有 CRI 都支持多个 RuntimeClass。
+RuntimeClass assumes a homogeneous node configuration across the cluster by default (which means
+that all nodes are configured the same way with respect to container runtimes). To support
+heterogenous node configurations, see [Scheduling](#scheduling) below.-->RuntimeClass 假设集群中的节点配置是同构的
+（换言之，所有的节点在容器运行时方面的配置是相同的）。
+如果需要支持异构节点，配置方法请参阅下面的 [Scheduling](#scheduling)。
+{{< /note >}}
 
 <!--
-RuntimeClass currently assumes a homogeneous node configuration across the cluster
-(which means that all nodes are configured the same way with respect to container runtimes). Any heterogeneity (varying configurations) must be
-managed independently of RuntimeClass through scheduling features
-(see [Assigning Pods to Nodes](/docs/concepts/configuration/assign-pod-node/)).
+The configurations have a corresponding `handler` name, referenced by the RuntimeClass. The
+handler must be a valid DNS 1123 label (alpha-numeric + `-` characters).
 -->
-{{< note >}}
-RuntimeClass 当前假设的是集群中的节点配置是同构的（换言之，所有的节点在容器运行时方面的配置是相同的）。
-任何异构性（不同的配置）必须通过调度功能在 RuntimeClass 之外单独管理（请参阅[在节点上分配 Pods](/docs/concepts/configuration/assign-pod-node/)）。
-{{< /note >}}
+所有这些配置都具有相应的 `handler` 名，并被 RuntimeClass 引用。
+handler 必须符合 DNS-1123 命名规范（字母、数字、或 `-`）。
 
 <!--
-The configurations have a corresponding `RuntimeHandler` name, referenced by the RuntimeClass. The
-RuntimeHandler must be a valid DNS-1123 subdomain (alpha-numeric characters, `-`, or `.`).
+#### 2. Create the corresponding RuntimeClass resources
 -->
-所有这些配置都具有相应的 `RuntimeHandler` 名，并被 RuntimeClass 引用。
-RuntimeHandler 必须是有效的 DNS-1123 子域（字母数字字符、`-` 或 `.`）。
+#### 2. 创建相应的 RuntimeClass 资源
 
 <!--
-#### 4. Create the corresponding RuntimeClass resources
+The configurations setup in step 1 should each have an associated `handler` name, which identifies
+the configuration. For each handler, create a corresponding RuntimeClass object.
 -->
-
-#### 4. 创建相应的 RuntimeClass 资源
+在上面步骤 1 中，每个配置都需要有一个用于标识配置的 `handler`。 
+针对每个 handler 需要创建一个 RuntimeClass 对象。
 
 <!--
-The configurations setup in step 3 should each have an associated `RuntimeHandler` name, which
-identifies the configuration. For each RuntimeHandler (and optionally the empty `""` handler),
-create a corresponding RuntimeClass object
-.-->
-步骤 3 中的配置设置应该有相应的 `RuntimeHandler` 名，用于标识配置。
-对应每个 RuntimeHandler（以及 `""` 所代表的空处理程序），都创建相应的 RuntimeClass 对象。
-
-<!--The RuntimeClass resource currently only has 2 significant fields: the RuntimeClass name
-(`metadata.name`) and the RuntimeHandler (`spec.runtimeHandler`). The object definition looks like this:
+The RuntimeClass resource currently only has 2 significant fields: the RuntimeClass name
+(`metadata.name`) and the handler (`handler`). The object definition looks like this:
 -->
-RuntimeClass 资源当前只有两个重要的字段：RuntimeClass 名 (`metadata.name`) 和 RuntimeHandler (`spec.runtimeHandler`)。
+RuntimeClass 资源当前只有两个重要的字段：RuntimeClass 名 (`metadata.name`) 和 handler (`handler`)。
 对象定义如下所示：
 
 ```yaml
-apiVersion: node.k8s.io/v1alpha1  # 在 node.k8s.io API 中对 RuntimeClass 进行定义
+apiVersion: node.k8s.io/v1beta1  # RuntimeClass is defined in the node.k8s.io API group
 kind: RuntimeClass
 metadata:
-  name: myclass  # 引用 RuntimeClass
-  # RuntimeClass 是不属于任何名字空间的资源
-spec:
-  runtimeHandler: myconfiguration  # 给出 CRI 配置的名称
+  name: myclass  # The name the RuntimeClass will be referenced by
+  # RuntimeClass is a non-namespaced resource
+handler: myconfiguration  # The name of the corresponding CRI configuration
 ```
 
+{{< note >}}
 <!--
 It is recommended that RuntimeClass write operations (create/update/patch/delete) be
 restricted to the cluster administrator. This is typically the default. See [Authorization
-Overview](/docs/reference/access-authn-authz/authorization/) for more details.
--->
-
-{{< note >}}
-建议将 RuntimeClass 写操作（create、update、patch 和 delete）限定于集群管理员使用。
+Overview](/docs/reference/access-authn-authz/authorization/) for more details.-->建议将 RuntimeClass 写操作（create、update、patch 和 delete）限定于集群管理员使用。
 通常这是默认配置。参阅[授权概述](/docs/reference/access-authn-authz/authorization/)了解更多信息。
 {{< /note >}}
 
 <!--
 ### Usage
-Once RuntimeClasses are configured for the cluster, using them is very simple. Specify a
-`runtimeClassName` in the Pod spec. For example:
 -->
-
 ### 使用说明
 
-一旦完成集群中 RuntimeClasses 的配置，使用起来非常简便。
+<!--
+Once RuntimeClasses are configured for the cluster, using them is very simple. Specify a
+`runtimeClassName` in the Pod spec. For example:
+-->
+一旦完成集群中 RuntimeClasses 的配置，使用起来非常方便。
 在 Pod spec 中指定 `runtimeClassName` 即可。例如:
 
 ```yaml
@@ -178,12 +170,13 @@ spec:
   # ...
 ```
 
-<!--This will instruct the Kubelet to use the named RuntimeClass to run this pod. If the named
+<!--
+This will instruct the Kubelet to use the named RuntimeClass to run this pod. If the named
 RuntimeClass does not exist, or the CRI cannot run the corresponding handler, the pod will enter the
 `Failed` terminal [phase](/docs/concepts/workloads/pods/pod-lifecycle/#pod-phase). Look for a
 corresponding [event](/docs/tasks/debug-application-cluster/debug-application-introspection/) for an
-error message.-->
-
+error message.
+-->
 这一设置会告诉 Kubelet 使用所指的 RuntimeClass 来运行该 pod。
 如果所指的 RuntimeClass 不存在或者 CRI 无法运行相应的 handler，那么 pod 将会进入 `Failed` 终止[阶段](/docs/concepts/workloads/pods/pod-lifecycle/#pod-phase)。
 你可以查看相应的[事件](/docs/tasks/debug-application-cluster/debug-application-introspection/)，获取出错信息。
@@ -194,4 +187,181 @@ to the behavior when the RuntimeClass feature is disabled.
 -->
 如果未指定 `runtimeClassName` ，则将使用默认的 RuntimeHandler，相当于禁用 RuntimeClass 功能特性。
 
+### CRI Configuration
+
+<!--
+For more details on setting up CRI runtimes, see [CRI installation](/docs/setup/production-environment/container-runtimes/).
+-->
+关于如何安装 CRI 运行时，请查阅[CRI installation](/docs/setup/production-environment/container-runtimes/)。
+
+#### dockershim
+
+<!--
+Kubernetes built-in dockershim CRI does not support runtime handlers.
+-->
+Kubernetes 内置 dockershim CRI 不支持配置运行时 handler。
+
+#### [containerd](https://containerd.io/)
+
+<!--
+Runtime handlers are configured through containerd's configuration at
+`/etc/containerd/config.toml`. Valid handlers are configured under the runtimes section:
+-->
+通过 containerd 的 `/etc/containerd/config.toml` 配置文件来配置运行时 handler。
+handler 需要配置在 runtimes 块中： 
+
+```
+[plugins.cri.containerd.runtimes.${HANDLER_NAME}]
+```
+
+<!--
+See containerd's config documentation for more details:
+https://github.com/containerd/cri/blob/master/docs/config.md
+-->
+更详细信息，请查阅 containerd 配置文档：
+https://github.com/containerd/cri/blob/master/docs/config.md
+
+#### [cri-o](https://cri-o.io/)
+
+<!--
+Runtime handlers are configured through cri-o's configuration at `/etc/crio/crio.conf`. Valid
+handlers are configured under the [crio.runtime
+table](https://github.com/kubernetes-sigs/cri-o/blob/master/docs/crio.conf.5.md#crioruntime-table):
+-->
+通过 cri-o 的 `/etc/crio/crio.conf` 配置文件来配置运行时 handler。
+handler 需要配置在[crio.runtime 表](https://github.com/kubernetes-sigs/cri-o/blob/master/docs/crio.conf.5.md#crioruntime-table)
+下方：
+
+```
+[crio.runtime.runtimes.${HANDLER_NAME}]
+  runtime_path = "${PATH_TO_BINARY}"
+```
+
+<!--
+See cri-o's config documentation for more details:
+https://github.com/kubernetes-sigs/cri-o/blob/master/cmd/crio/config.go
+-->
+更详细信息，请查阅 containerd 配置文档：
+https://github.com/kubernetes-sigs/cri-o/blob/master/cmd/crio/config.go
+
+### Scheduling
+
+{{< feature-state for_k8s_version="v1.16" state="beta" >}}
+
+<!--
+As of Kubernetes v1.16, RuntimeClass includes support for heterogenous clusters through its
+`scheduling` fields. Through the use of these fields, you can ensure that pods running with this
+RuntimeClass are scheduled to nodes that support it. To use the scheduling support, you must have
+the RuntimeClass [admission controller][] enabled (the default, as of 1.16).
+-->
+在 Kubernetes v1.16 版本里，RuntimeClass 特性引入了 `scheduling` 字段来支持异构集群。
+通过该字段，可以确保 pod 被调度到支持指定运行时的节点上。
+该调度支持，需要确保 RuntimeClass [admission controller][] 处于开启状态（1.16 版本默认开启）。
+
+<!--
+To ensure pods land on nodes supporting a specific RuntimeClass, that set of nodes should have a
+common label which is then selected by the `runtimeclass.scheduling.nodeSelector` field. The
+RuntimeClass's nodeSelector is merged with the pod's nodeSelector in admission, effectively taking
+the intersection of the set of nodes selected by each. If there is a conflict, the pod will be
+rejected.
+-->
+为了确保 pod 会被调度到支持指定运行时的 node 上，每个 node 需要设置一个通用的 label 用于被 
+`runtimeclass.scheduling.nodeSelector` 挑选。在 admission 阶段，RuntimeClass 的 nodeSelector 将会于
+pod 的 nodeSelector 合并，取二者的交集。如果有冲突，pod 将会被拒绝。
+
+<!--
+If the supported nodes are tainted to prevent other RuntimeClass pods from running on the node, you
+can add `tolerations` to the RuntimeClass. As with the `nodeSelector`, the tolerations are merged
+with the pod's tolerations in admission, effectively taking the union of the set of nodes tolerated
+by each.
+-->
+如果 node 需要阻止某些需要特定 RuntimeClass 的 pod，可以在 `tolerations` 中指定。
+与 `nodeSelector` 一样，tolerations 也在 admission 阶段与 pod 的 tolerations 合并，取二者的并集。
+
+<!--
+To learn more about configuring the node selector and tolerations, see [Assigning Pods to
+Nodes](/docs/concepts/configuration/assign-pod-node/).
+-->
+更多有关 node selector 和 tolerations 的配置信息，请查阅 
+[Assigning Pods to Nodes](/docs/concepts/configuration/assign-pod-node/)。
+
+[admission controller]: /docs/reference/access-authn-authz/admission-controllers/
+
+### Pod Overhead
+
+{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
+
+<!--
+As of Kubernetes v1.16, RuntimeClass includes support for specifying overhead associated with
+running a pod, as part of the [`PodOverhead`](/docs/concepts/configuration/pod-overhead.md) feature.
+To use `PodOverhead`, you must have the PodOverhead [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
+enabled (it is off by default).
+-->
+在 Kubernetes v1.16 版本中，RuntimeClass 开始支持 pod 的 overhead，作为 [`PodOverhead`](/docs/concepts/configuration/pod-overhead.md) 
+特性的一部分。
+若要使用 `PodOverhead` 特性，你需要确保 PodOverhead 特性开关处于开启状态（默认为关闭状态）。
+
+<!--
+Pod overhead is defined in RuntimeClass through the `Overhead` fields. Through the use of these fields,
+you can specify the overhead of running pods utilizing this RuntimeClass and ensure these overheads
+are accounted for in Kubernetes.
+-->
+Pod 的 overhead 在 RuntimeClass 的 `Overhead` 字段定义，该字段用于指定使用 RuntimeClass 特性时带来的 overhead。
+
+### Upgrading RuntimeClass from Alpha to Beta
+
+<!--
+The RuntimeClass Beta feature includes the following changes:
+-->
+RuntimeClass Beta 特性包含如下几个改变：
+
+<!--
+- The `node.k8s.io` API group and `runtimeclasses.node.k8s.io` resource have been migrated to a
+  built-in API from a CustomResourceDefinition.
+- The `spec` has been inlined in the RuntimeClass definition (i.e. there is no more
+  RuntimeClassSpec).
+- The `runtimeHandler` field has been renamed `handler`.
+- The `handler` field is now required in all API versions. This means the `runtimeHandler` field in
+  the Alpha API is also required.
+- The `handler` field must be a valid DNS label ([RFC 1123](https://tools.ietf.org/html/rfc1123)),
+  meaning it can no longer contain `.` characters (in all versions). Valid handlers match the
+  following regular expression: `^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`.
+-->
+- `node.k8s.io` API 组和 `runtimeclasses.node.k8s.io` 资源已从 CRD 中迁移到内置的 API 中；
+- `spec` 被放置到 RuntimeClass 中（例如，没有 RuntimeClassSpec 了）；
+- `runtimeHandler` 字段重命名为 `handler`；
+- `handler` 字段需要在所有版本的 API 提供，这意味着 `runtimeHandler` 字段在 Alpha API 中也需要提供；
+- `handler` 字段必须是一个合法的 DNS 标识([RFC 1123](https://tools.ietf.org/html/rfc1123))，
+  这意味着不可以包含 `.` 字符。合法的 handler 必须满足如下规则：`^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`。
+
+<!--
+**Action Required:** The following actions are required to upgrade from the alpha version of the
+RuntimeClass feature to the beta version:
+-->
+**Action Required:** RuntimeClass 特性从 alpha 版本升级到 beta 版本，需要做如下动作：
+
+<!--
+- RuntimeClass resources must be recreated *after* upgrading to v1.14, and the
+  `runtimeclasses.node.k8s.io` CRD should be manually deleted:
+  ```
+  kubectl delete customresourcedefinitions.apiextensions.k8s.io runtimeclasses.node.k8s.io
+  ```
+- Alpha RuntimeClasses with an unspecified or empty `runtimeHandler` or those using a `.` character
+  in the handler are no longer valid, and must be migrated to a valid handler configuration (see
+  above).
+-->
+- RuntimeClass 资源必须在升级到 v1.14 *之后* 再创建，并且 CRD 资源 `runtimeclasses.node.k8s.io` 必须要手动删除：
+  ```
+  kubectl delete customresourcedefinitions.apiextensions.k8s.io runtimeclasses.node.k8s.io
+  ```
+- RuntimeClasses 中未指定或为空的 `runtimeHandler` 和 使用包含 `.` 符号的 handler 将不再合法，
+  必须迁移成合法的 handler 配置（见上）。
+  
+### Further Reading
+
+- [RuntimeClass Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/runtime-class.md)
+- [RuntimeClass Scheduling Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/runtime-class-scheduling.md)
+- Read about the [Pod Overhead](/docs/concepts/configuration/pod-overhead/) concept
+- [PodOverhead Feature Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/20190226-pod-overhead.md)
+
 {{% /capture %}}