Skip to content

Commit

Permalink
feat: Add TopologySpreadConstraints support to the humiocluster CRD
Browse files Browse the repository at this point in the history 
This PR fixes #342 with the addition of TopologySpreadConstraints to the CR. Node instances of operator deployed by helm will not automatically update the CR for those instances the CR must be applied manually
  • Loading branch information
@ryanfaircloth
ryanfaircloth committed Feb 15, 2023
1 parent f902fd1 commit 1c25d7c
Show file tree
Hide file tree
Showing 6 changed files with 441 additions and 0 deletions.
3 changes: 3 additions & 0 deletions api/v1alpha1/humiocluster_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,9 @@ type HumioNodeSpec struct {
// Tolerations defines the tolerations that will be attached to the humio pods
Tolerations []corev1.Toleration `json:"tolerations,omitempty"`

// TopologySpreadConstraints defines the topologySpreadConstraints that will be attached to the humio pods
TopologySpreadConstraints []corev1.TopologySpreadConstraint `json:"topologySpreadConstraints,omitempty"`

// SidecarContainers can be used in advanced use-cases where you want one or more sidecar container added to the
// Humio pod to help out in debugging purposes.
SidecarContainers []corev1.Container `json:"sidecarContainer,omitempty"`
Expand Down
7 changes: 7 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

211 changes: 211 additions & 0 deletions charts/humio-operator/crds/core.humio.com_humioclusters.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11938,6 +11938,116 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
description: TopologySpreadConstraints defines the topologySpreadConstraints
that will be attached to the humio pods
items:
description: TopologySpreadConstraint specifies how to
spread matching pods among the given topology.
properties:
labelSelector:
description: LabelSelector is used to find matching
pods. Pods that match this label selector are counted
to determine the number of pods in their corresponding
topology domain.
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: A label selector requirement is
a selector that contains values, a key, and
an operator that relates the key and values.
properties:
key:
description: key is the label key that the
selector applies to.
type: string
operator:
description: operator represents a key's
relationship to a set of values. Valid
operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string
values. If the operator is In or NotIn,
the values array must be non-empty. If
the operator is Exists or DoesNotExist,
the values array must be empty. This array
is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value}
pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions,
whose key field is "key", the operator is "In",
and the values array contains only "value".
The requirements are ANDed.
type: object
type: object
maxSkew:
description: 'MaxSkew describes the degree to which
pods may be unevenly distributed. When `whenUnsatisfiable=DoNotSchedule`,
it is the maximum permitted difference between the
number of matching pods in the target topology and
the global minimum. For example, in a 3-zone cluster,
MaxSkew is set to 1, and pods with the same labelSelector
spread as 1/1/0: | zone1 | zone2 | zone3 | | P | P | |
- if MaxSkew is 1, incoming pod can only be scheduled
to zone3 to become 1/1/1; scheduling it onto zone1(zone2)
would make the ActualSkew(2-0) on zone1(zone2) violate
MaxSkew(1). - if MaxSkew is 2, incoming pod can
be scheduled onto any zone. When `whenUnsatisfiable=ScheduleAnyway`,
it is used to give higher precedence to topologies
that satisfy it. It''s a required field. Default
value is 1 and 0 is not allowed.'
format: int32
type: integer
topologyKey:
description: TopologyKey is the key of node labels.
Nodes that have a label with this key and identical
values are considered to be in the same topology.
We consider each <key, value> as a "bucket", and
try to put balanced number of pods into each bucket.
It's a required field.
type: string
whenUnsatisfiable:
description: 'WhenUnsatisfiable indicates how to deal
with a pod if it doesn''t satisfy the spread constraint.
- DoNotSchedule (default) tells the scheduler not
to schedule it. - ScheduleAnyway tells the scheduler
to schedule the pod in any location, but giving
higher precedence to topologies that would help
reduce the skew. A constraint is considered "Unsatisfiable"
for an incoming pod if and only if every possible
node assignment for that pod would violate "MaxSkew"
on some topology. For example, in a 3-zone cluster,
MaxSkew is set to 1, and pods with the same labelSelector
spread as 3/1/1: | zone1 | zone2 | zone3 | | P P
P | P | P | If WhenUnsatisfiable is set
to DoNotSchedule, incoming pod can only be scheduled
to zone2(zone3) to become 3/2/1(3/1/2) as ActualSkew(2-1)
on zone2(zone3) satisfies MaxSkew(1). In other words,
the cluster can still be imbalanced, but scheduler
won''t make it *more* imbalanced. It''s a required
field.'
type: string
required:
- maxSkew
- topologyKey
- whenUnsatisfiable
type: object
type: array
updateStrategy:
description: UpdateStrategy controls how Humio pods are
updated when changes are made to the HumioCluster resource
Expand Down Expand Up @@ -13490,6 +13600,107 @@ spec:
type: string
type: object
type: array
topologySpreadConstraints:
description: TopologySpreadConstraints defines the topologySpreadConstraints
that will be attached to the humio pods
items:
description: TopologySpreadConstraint specifies how to spread matching
pods among the given topology.
properties:
labelSelector:
description: LabelSelector is used to find matching pods. Pods
that match this label selector are counted to determine the
number of pods in their corresponding topology domain.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: operator represents a key's relationship
to a set of values. Valid operators are In, NotIn,
Exists and DoesNotExist.
type: string
values:
description: values is an array of string values.
If the operator is In or NotIn, the values array
must be non-empty. If the operator is Exists or
DoesNotExist, the values array must be empty. This
array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field is
"key", the operator is "In", and the values array contains
only "value". The requirements are ANDed.
type: object
type: object
maxSkew:
description: 'MaxSkew describes the degree to which pods may
be unevenly distributed. When `whenUnsatisfiable=DoNotSchedule`,
it is the maximum permitted difference between the number
of matching pods in the target topology and the global minimum.
For example, in a 3-zone cluster, MaxSkew is set to 1, and
pods with the same labelSelector spread as 1/1/0: | zone1
| zone2 | zone3 | | P | P | | - if MaxSkew is
1, incoming pod can only be scheduled to zone3 to become 1/1/1;
scheduling it onto zone1(zone2) would make the ActualSkew(2-0)
on zone1(zone2) violate MaxSkew(1). - if MaxSkew is 2, incoming
pod can be scheduled onto any zone. When `whenUnsatisfiable=ScheduleAnyway`,
it is used to give higher precedence to topologies that satisfy
it. It''s a required field. Default value is 1 and 0 is not
allowed.'
format: int32
type: integer
topologyKey:
description: TopologyKey is the key of node labels. Nodes that
have a label with this key and identical values are considered
to be in the same topology. We consider each <key, value>
as a "bucket", and try to put balanced number of pods into
each bucket. It's a required field.
type: string
whenUnsatisfiable:
description: 'WhenUnsatisfiable indicates how to deal with a
pod if it doesn''t satisfy the spread constraint. - DoNotSchedule
(default) tells the scheduler not to schedule it. - ScheduleAnyway
tells the scheduler to schedule the pod in any location, but
giving higher precedence to topologies that would help reduce
the skew. A constraint is considered "Unsatisfiable" for
an incoming pod if and only if every possible node assignment
for that pod would violate "MaxSkew" on some topology. For
example, in a 3-zone cluster, MaxSkew is set to 1, and pods
with the same labelSelector spread as 3/1/1: | zone1 | zone2
| zone3 | | P P P | P | P | If WhenUnsatisfiable is
set to DoNotSchedule, incoming pod can only be scheduled to
zone2(zone3) to become 3/2/1(3/1/2) as ActualSkew(2-1) on
zone2(zone3) satisfies MaxSkew(1). In other words, the cluster
can still be imbalanced, but scheduler won''t make it *more*
imbalanced. It''s a required field.'
type: string
required:
- maxSkew
- topologyKey
- whenUnsatisfiable
type: object
type: array
updateStrategy:
description: UpdateStrategy controls how Humio pods are updated when
changes are made to the HumioCluster resource that results in a
Expand Down
Loading

0 comments on commit 1c25d7c

Please sign in to comment.