Skip to content

Commit

Permalink
Merge pull request #3598 from AshleyDumaine/add-pod-topology-spread-c…
Browse files Browse the repository at this point in the history 
…onstraints

Support for Pod Topology Spread Constraints for Prometheus, Alertmanager, and ThanosRuler CRDs
  • Loading branch information
@brancz
brancz committed Oct 19, 2020
2 parents fd07e4c + c31bc17 commit 3f9aa94
Show file tree
Hide file tree
Showing 14 changed files with 600 additions and 3 deletions.
3 changes: 3 additions & 0 deletions Documentation/api.md
View file
Expand Up @@ -171,6 +171,7 @@ AlertmanagerSpec is a specification of the desired behavior of the Alertmanager
| resources | Define resources requests and limits for single Pods. | [v1.ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.17/#resourcerequirements-v1-core) | false |
| affinity | If specified, the pod's scheduling constraints. | *v1.Affinity | false |
| tolerations | If specified, the pod's tolerations. | []v1.Toleration | false |
| topologySpreadConstraints | If specified, the pod's topology spread constraints. | []v1.TopologySpreadConstraint | false |
| securityContext | SecurityContext holds pod-level security attributes and common container settings. This defaults to the default PodSecurityContext. | *v1.PodSecurityContext | false |
| serviceAccountName | ServiceAccountName is the name of the ServiceAccount to use to run the Prometheus Pods. | string | false |
| listenLocal | ListenLocal makes the Alertmanager server listen on loopback, so that it does not bind against the Pod IP. Note this is only for the Alertmanager UI, not the gossip communication. | bool | false |
Expand Down Expand Up @@ -558,6 +559,7 @@ PrometheusSpec is a specification of the desired behavior of the Prometheus clus
| configMaps | ConfigMaps is a list of ConfigMaps in the same namespace as the Prometheus object, which shall be mounted into the Prometheus Pods. The ConfigMaps are mounted into /etc/prometheus/configmaps/<configmap-name>. | []string | false |
| affinity | If specified, the pod's scheduling constraints. | *v1.Affinity | false |
| tolerations | If specified, the pod's tolerations. | []v1.Toleration | false |
| topologySpreadConstraints | If specified, the pod's topology spread constraints. | []v1.TopologySpreadConstraint | false |
| remoteWrite | If specified, the remote_write spec. This is an experimental feature, it may change in any upcoming release in a breaking way. | [][RemoteWriteSpec](#remotewritespec) | false |
| remoteRead | If specified, the remote_read spec. This is an experimental feature, it may change in any upcoming release in a breaking way. | [][RemoteReadSpec](#remotereadspec) | false |
| securityContext | SecurityContext holds pod-level security attributes and common container settings. This defaults to the default PodSecurityContext. | *v1.PodSecurityContext | false |
Expand Down Expand Up @@ -895,6 +897,7 @@ ThanosRulerSpec is a specification of the desired behavior of the ThanosRuler. M
| resources | Resources defines the resource requirements for single Pods. If not provided, no requests/limits will be set | [v1.ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.17/#resourcerequirements-v1-core) | false |
| affinity | If specified, the pod's scheduling constraints. | *v1.Affinity | false |
| tolerations | If specified, the pod's tolerations. | []v1.Toleration | false |
| topologySpreadConstraints | If specified, the pod's topology spread constraints. | []v1.TopologySpreadConstraint | false |
| securityContext | SecurityContext holds pod-level security attributes and common container settings. This defaults to the default PodSecurityContext. | *v1.PodSecurityContext | false |
| priorityClassName | Priority class assigned to the Pods | string | false |
| serviceAccountName | ServiceAccountName is the name of the ServiceAccount to use to run the Thanos Ruler Pods. | string | false |
Expand Down
282 changes: 282 additions & 0 deletions bundle.yaml
View file

Large diffs are not rendered by default.

94 changes: 94 additions & 0 deletions example/prometheus-operator-crd/monitoring.coreos.com_alertmanagers.yaml
View file
Expand Up @@ -3429,6 +3429,100 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
description: If specified, the pod's topology spread constraints.
items:
description: TopologySpreadConstraint specifies how to spread matching
pods among the given topology.
properties:
labelSelector:
description: LabelSelector is used to find matching pods. Pods
that match this label selector are counted to determine the
number of pods in their corresponding topology domain.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: operator represents a key's relationship
to a set of values. Valid operators are In, NotIn,
Exists and DoesNotExist.
type: string
values:
description: values is an array of string values.
If the operator is In or NotIn, the values array
must be non-empty. If the operator is Exists or
DoesNotExist, the values array must be empty. This
array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field is
"key", the operator is "In", and the values array contains
only "value". The requirements are ANDed.
type: object
type: object
maxSkew:
description: 'MaxSkew describes the degree to which pods may
be unevenly distributed. It''s the maximum permitted difference
between the number of matching pods in any two topology domains
of a given topology type. For example, in a 3-zone cluster,
MaxSkew is set to 1, and pods with the same labelSelector
spread as 1/1/0: | zone1 | zone2 | zone3 | | P | P | |
- if MaxSkew is 1, incoming pod can only be scheduled to zone3
to become 1/1/1; scheduling it onto zone1(zone2) would make
the ActualSkew(2-0) on zone1(zone2) violate MaxSkew(1). -
if MaxSkew is 2, incoming pod can be scheduled onto any zone.
It''s a required field. Default value is 1 and 0 is not allowed.'
format: int32
type: integer
topologyKey:
description: TopologyKey is the key of node labels. Nodes that
have a label with this key and identical values are considered
to be in the same topology. We consider each <key, value>
as a "bucket", and try to put balanced number of pods into
each bucket. It's a required field.
type: string
whenUnsatisfiable:
description: 'WhenUnsatisfiable indicates how to deal with a
pod if it doesn''t satisfy the spread constraint. - DoNotSchedule
(default) tells the scheduler not to schedule it - ScheduleAnyway
tells the scheduler to still schedule it It''s considered
as "Unsatisfiable" if and only if placing incoming pod on
any topology violates "MaxSkew". For example, in a 3-zone
cluster, MaxSkew is set to 1, and pods with the same labelSelector
spread as 3/1/1: | zone1 | zone2 | zone3 | | P P P | P | P |
If WhenUnsatisfiable is set to DoNotSchedule, incoming pod
can only be scheduled to zone2(zone3) to become 3/2/1(3/1/2)
as ActualSkew(2-1) on zone2(zone3) satisfies MaxSkew(1). In
other words, the cluster can still be imbalanced, but scheduler
won''t make it *more* imbalanced. It''s a required field.'
type: string
required:
- maxSkew
- topologyKey
- whenUnsatisfiable
type: object
type: array
version:
description: Version the cluster should be on.
type: string
Expand Down
94 changes: 94 additions & 0 deletions example/prometheus-operator-crd/monitoring.coreos.com_prometheuses.yaml
View file
Expand Up @@ -5033,6 +5033,100 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
description: If specified, the pod's topology spread constraints.
items:
description: TopologySpreadConstraint specifies how to spread matching
pods among the given topology.
properties:
labelSelector:
description: LabelSelector is used to find matching pods. Pods
that match this label selector are counted to determine the
number of pods in their corresponding topology domain.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: operator represents a key's relationship
to a set of values. Valid operators are In, NotIn,
Exists and DoesNotExist.
type: string
values:
description: values is an array of string values.
If the operator is In or NotIn, the values array
must be non-empty. If the operator is Exists or
DoesNotExist, the values array must be empty. This
array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field is
"key", the operator is "In", and the values array contains
only "value". The requirements are ANDed.
type: object
type: object
maxSkew:
description: 'MaxSkew describes the degree to which pods may
be unevenly distributed. It''s the maximum permitted difference
between the number of matching pods in any two topology domains
of a given topology type. For example, in a 3-zone cluster,
MaxSkew is set to 1, and pods with the same labelSelector
spread as 1/1/0: | zone1 | zone2 | zone3 | | P | P | |
- if MaxSkew is 1, incoming pod can only be scheduled to zone3
to become 1/1/1; scheduling it onto zone1(zone2) would make
the ActualSkew(2-0) on zone1(zone2) violate MaxSkew(1). -
if MaxSkew is 2, incoming pod can be scheduled onto any zone.
It''s a required field. Default value is 1 and 0 is not allowed.'
format: int32
type: integer
topologyKey:
description: TopologyKey is the key of node labels. Nodes that
have a label with this key and identical values are considered
to be in the same topology. We consider each <key, value>
as a "bucket", and try to put balanced number of pods into
each bucket. It's a required field.
type: string
whenUnsatisfiable:
description: 'WhenUnsatisfiable indicates how to deal with a
pod if it doesn''t satisfy the spread constraint. - DoNotSchedule
(default) tells the scheduler not to schedule it - ScheduleAnyway
tells the scheduler to still schedule it It''s considered
as "Unsatisfiable" if and only if placing incoming pod on
any topology violates "MaxSkew". For example, in a 3-zone
cluster, MaxSkew is set to 1, and pods with the same labelSelector
spread as 3/1/1: | zone1 | zone2 | zone3 | | P P P | P | P |
If WhenUnsatisfiable is set to DoNotSchedule, incoming pod
can only be scheduled to zone2(zone3) to become 3/2/1(3/1/2)
as ActualSkew(2-1) on zone2(zone3) satisfies MaxSkew(1). In
other words, the cluster can still be imbalanced, but scheduler
won''t make it *more* imbalanced. It''s a required field.'
type: string
required:
- maxSkew
- topologyKey
- whenUnsatisfiable
type: object
type: array
version:
description: Version of Prometheus to be deployed.
type: string
Expand Down

0 comments on commit 3f9aa94

Please sign in to comment.