Skip to content
Permalink
Browse files

Providing more detail on EndpointSlice implementation (#18343)

  • Loading branch information
robscott authored and k8s-ci-robot committed Jan 14, 2020
1 parent f41978d commit c1921541ef687a8dd95574f83a6cb34326759904
@@ -1,9 +1,9 @@
---
reviewers:
- freehan
title: Endpoint Slices
title: EndpointSlices
feature:
title: Endpoint Slices
title: EndpointSlices
description: >
Scalable tracking of network endpoints in a Kubernetes cluster.
@@ -16,21 +16,22 @@ weight: 10

{{< feature-state for_k8s_version="v1.17" state="beta" >}}

_Endpoint Slices_ provide a simple way to track network endpoints within a
_EndpointSlices_ provide a simple way to track network endpoints within a
Kubernetes cluster. They offer a more scalable and extensible alternative to
Endpoints.

{{% /capture %}}

{{% capture body %}}

## Endpoint Slice resources {#endpointslice-resource}
## EndpointSlice resources {#endpointslice-resource}

In Kubernetes, an EndpointSlice contains references to a set of network
endpoints. The EndpointSlice controller automatically creates Endpoint Slices
for a Kubernetes Service when a selector is specified. These Endpoint Slices
will include references to any Pods that match the Service selector. Endpoint
Slices group network endpoints together by unique Service and Port combinations.
endpoints. The EndpointSlice controller automatically creates EndpointSlices
for a Kubernetes Service when a {{< glossary_tooltip text="selector"
term_id="selector" >}} is specified. These EndpointSlices will include
references to any Pods that match the Service selector. EndpointSlices group
network endpoints together by unique Service and Port combinations.

As an example, here's a sample EndpointSlice resource for the `example`
Kubernetes Service.
@@ -58,22 +59,110 @@ endpoints:
topology.kubernetes.io/zone: us-west2-a
```

By default, Endpoint Slices managed by the EndpointSlice controller will have no
more than 100 endpoints each. Below this scale, Endpoint Slices should map 1:1
By default, EndpointSlices managed by the EndpointSlice controller will have no
more than 100 endpoints each. Below this scale, EndpointSlices should map 1:1
with Endpoints and Services and have similar performance.

Endpoint Slices can act as the source of truth for kube-proxy when it comes to
EndpointSlices can act as the source of truth for kube-proxy when it comes to
how to route internal traffic. When enabled, they should provide a performance
improvement for services with large numbers of endpoints.

## Address Types
### Address Types

EndpointSlices support three address types:

* IPv4
* IPv6
* FQDN (Fully Qualified Domain Name)

### Topology

Each endpoint within an EndpointSlice can contain relevant topology information.
This is used to indicate where an endpoint is, containing information about the
corresponding Node, zone, and region. When the values are available, the
following Topology labels will be set by the EndpointSlice controller:

* `kubernetes.io/hostname` - The name of the Node this endpoint is on.
* `topology.kubernetes.io/zone` - The zone this endpoint is in.
* `topology.kubernetes.io/region` - The region this endpoint is in.

The values of these labels are derived from resources associated with each
endpoint in a slice. The hostname label represents the value of the NodeName
field on the corresponding Pod. The zone and region labels represent the value
of the labels with the same names on the corresponding Node.

### Management

By default, EndpointSlices are created and managed by the EndpointSlice
controller. There are a variety of other use cases for EndpointSlices, such as
service mesh implementations, that could result in other entities or controllers
managing additional sets of EndpointSlices. To ensure that multiple entities can
manage EndpointSlices without interfering with each other, a
`endpointslice.kubernetes.io/managed-by` label is used to indicate the entity
managing an EndpointSlice. The EndpointSlice controller sets
`endpointslice-controller.k8s.io` as the value for this label on all
EndpointSlices it manages. Other entities managing EndpointSlices should also
set a unique value for this label.

### Ownership

In most use cases, EndpointSlices will be owned by the Service that it tracks
endpoints for. This is indicated by an owner reference on each EndpointSlice as
well as a `kubernetes.io/service-name` label that enables simple lookups of all
EndpointSlices belonging to a Service.

## EndpointSlice Controller

The EndpointSlice controller watches Services and Pods to ensure corresponding
EndpointSlices are up to date. The controller will manage EndpointSlices for
every Service with a selector specified. These will represent the IPs of Pods
matching the Service selector.

### Size of EndpointSlices

By default, EndpointSlices are limited to a size of 100 endpoints each. You can
configure this with the `--max-endpoints-per-slice` {{< glossary_tooltip
text="kube-controller-manager" term_id="kube-controller-manager" >}} flag up to
a maximum of 1000.

### Distribution of EndpointSlices

Each EndpointSlice has a set of ports that applies to all endpoints within the
resource. When named ports are used for a Service, Pods may end up with
different target port numbers for the same named port, requiring different
EndpointSlices. This is similar to the logic behind how subsets are grouped
with Endpoints.

The controller tries to fill EndpointSlices as full as possible, but does not
actively rebalance them. The logic of the controller is fairly straightforward:

1. Iterate through existing EndpointSlices, remove endpoints that are no longer
desired and update matching endpoints that have changed.
2. Iterate through EndpointSlices that have been modified in the first step and
fill them up with any new endpoints needed.
3. If there's still new endpoints left to add, try to fit them into a previously
unchanged slice and/or create new ones.

Importantly, the third step prioritizes limiting EndpointSlice updates over a
perfectly full distribution of EndpointSlices. As an example, if there are 10
new endpoints to add and 2 EndpointSlices with room for 5 more endpoints each,
this approach will create a new EndpointSlice instead of filling up the 2
existing EndpointSlices. In other words, a single EndpointSlice creation is
preferrable to multiple EndpointSlice updates.

With kube-proxy running on each Node and watching EndpointSlices, every change
to an EndpointSlice becomes relatively expensive since it will be transmitted to
every Node in the cluster. This approach is intended to limit the number of
changes that need to be sent to every Node, even if it may result with multiple
EndpointSlices that are not full.

In practice, this less than ideal distribution should be rare. Most changes
processed by the EndpointSlice controller will be small enough to fit in an
existing EndpointSlice, and if not, a new EndpointSlice is likely going to be
necessary soon anyway. Rolling updates of Deployments also provide a natural
repacking of EndpointSlices with all pods and their corresponding endpoints
getting replaced.

## Motivation

The Endpoints API has provided a simple and straightforward way of
@@ -86,14 +175,14 @@ Since all network endpoints for a Service were stored in a single Endpoints
resource, those resources could get quite large. That affected the performance
of Kubernetes components (notably the master control plane) and resulted in
significant amounts of network traffic and processing when Endpoints changed.
Endpoint Slices help you mitigate those issues as well as provide an extensible
EndpointSlices help you mitigate those issues as well as provide an extensible
platform for additional features such as topological routing.

{{% /capture %}}

{{% capture whatsnext %}}

* [Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices)
* [Enabling EndpointSlices](/docs/tasks/administer-cluster/enabling-endpointslices)
* Read [Connecting Applications with Services](/docs/concepts/services-networking/connect-applications-service/)

{{% /capture %}}
@@ -184,18 +184,18 @@ An ExternalName Service is a special case of Service that does not have
selectors and uses DNS names instead. For more information, see the
[ExternalName](#externalname) section later in this document.

### Endpoint Slices
### EndpointSlices
{{< feature-state for_k8s_version="v1.17" state="beta" >}}

Endpoint Slices are an API resource that can provide a more scalable alternative
to Endpoints. Although conceptually quite similar to Endpoints, Endpoint Slices
EndpointSlices are an API resource that can provide a more scalable alternative
to Endpoints. Although conceptually quite similar to Endpoints, EndpointSlices
allow for distributing network endpoints across multiple resources. By default,
an Endpoint Slice is considered "full" once it reaches 100 endpoints, at which
point additional Endpoint Slices will be created to store any additional
an EndpointSlice is considered "full" once it reaches 100 endpoints, at which
point additional EndpointSlices will be created to store any additional
endpoints.

Endpoint Slices provide additional attributes and functionality which is
described in detail in [Endpoint Slices](/docs/concepts/services-networking/endpoint-slices/).
EndpointSlices provide additional attributes and functionality which is
described in detail in [EndpointSlices](/docs/concepts/services-networking/endpoint-slices/).

## Virtual IPs and service proxies

@@ -1196,6 +1196,6 @@ which encompass the current ClusterIP, NodePort, and LoadBalancer modes and more

* Read [Connecting Applications with Services](/docs/concepts/services-networking/connect-applications-service/)
* Read about [Ingress](/docs/concepts/services-networking/ingress/)
* Read about [Endpoint Slices](/docs/concepts/services-networking/endpoint-slices/)
* Read about [EndpointSlices](/docs/concepts/services-networking/endpoint-slices/)

{{% /capture %}}
@@ -1,5 +1,5 @@
---
title: Endpoint Slice
title: EndpointSlice
id: endpoint-slice
date: 2018-04-12
full_link: /docs/concepts/services-networking/endpoint-slices/
@@ -2,12 +2,12 @@
reviewers:
- bowei
- freehan
title: Enabling Endpoint Slices
title: Enabling EndpointSlices
content_template: templates/task
---

{{% capture overview %}}
This page provides an overview of enabling Endpoint Slices in Kubernetes.
This page provides an overview of enabling EndpointSlices in Kubernetes.
{{% /capture %}}


@@ -19,30 +19,48 @@ This page provides an overview of enabling Endpoint Slices in Kubernetes.

## Introduction

Endpoint Slices provide a scalable and extensible alternative to Endpoints in
EndpointSlices provide a scalable and extensible alternative to Endpoints in
Kubernetes. They build on top of the base of functionality provided by Endpoints
and extend that in a scalable way. When Services have a large number (>100) of
network endpoints, they will be split into multiple smaller Endpoint Slice
network endpoints, they will be split into multiple smaller EndpointSlice
resources instead of a single large Endpoints resource.

## Enabling Endpoint Slices
## Enabling EndpointSlices

{{< feature-state for_k8s_version="v1.17" state="beta" >}}

{{< note >}}
Although Endpoint Slices may eventually replace Endpoints, many Kubernetes
components still rely on Endpoints. For now, enabling Endpoint Slices should be
Although EndpointSlices may eventually replace Endpoints, many Kubernetes
components still rely on Endpoints. For now, enabling EndpointSlices should be
seen as an addition to Endpoints in a cluster, not a replacement for them.
{{< /note >}}

As an alpha feature, Endpoint Slices are not enabled by default in Kubernetes.
To enable them, the EndpointSlice feature gate will need to be enabled
(`--feature-gates=EndpointSlice=true`).
EndpointSlices are considered a beta feature, but only the API is enabled by
default. Both the EndpointSlice controller and the usage of EndpointSlices by
kube-proxy are not enabled by default.

## Using Endpoint Slices
The EndpointSlice controller creates and manages EndpointSlices in a cluster.
You can enable it with the `EndpointSlice` [feature
gate](/docs/reference/command-line-tools-reference/feature-gates/) on the {{<
glossary_tooltip text="kube-apiserver" term_id="kube-apiserver" >}} and {{<
glossary_tooltip text="kube-controller-manager"
term_id="kube-controller-manager" >}} (`--feature-gates=EndpointSlice=true`).

With Endpoint Slices fully enabled in your cluster, you should see corresponding
For better scalability, you can also enable this feature gate on {{<
glossary_tooltip text="kube-proxy" term_id="kube-proxy" >}} so EndpointSlices
will be used as the data source instead of Endpoints.

## Using EndpointSlices

With EndpointSlices fully enabled in your cluster, you should see corresponding
EndpointSlice resources for each Endpoints resource. In addition to supporting
existing Endpoints functionality, Endpoint Slices should include new bits of
existing Endpoints functionality, EndpointSlices should include new bits of
information such as topology. They will allow for greater scalability and
extensibility of network endpoints in your cluster.
extensibility of network endpoints in your cluster.

{{% capture whatsnext %}}

* Read about [EndpointSlices](/docs/concepts/services-networking/endpoint-slices/)
* Read [Connecting Applications with Services](/docs/concepts/services-networking/connect-applications-service/)

{{% /capture %}}

0 comments on commit c192154

Please sign in to comment.
You can’t perform that action at this time.