Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

README: Add note about cluster-autoscaler not supporting multiple AZs #647

Merged
merged 2 commits into from Mar 25, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
38 changes: 37 additions & 1 deletion README.md
Expand Up @@ -130,7 +130,7 @@ To use a 3-5 node Auto Scaling Group, run:
eksctl create cluster --name=cluster-5 --nodes-min=3 --nodes-max=5
```

> NOTE: You will still need to install and configure autoscaling. See the "Enable Autoscaling" section below.
> NOTE: You will still need to install and configure autoscaling. See the "Enable Autoscaling" section below. Also note that depending on your workloads you might need to use a separate nodegroup for each AZ. See [Zone-aware Autoscaling](#zone-aware-autoscaling) below for more info.

To use 30 `c4.xlarge` nodes and prevent updating current context in `~/.kube/config`, run:

Expand Down Expand Up @@ -410,6 +410,42 @@ and `k8s.io/cluster-autoscaler/<clusterName>` tags, so nodegroup discovery shoul

[cluster autoscaler]: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md

#### Zone-aware Autoscaling

If your workloads are zone-specific you'll need to create separate nodegroups for each zone. This is because the `cluster-autoscaler` assumes that all nodes in a group are exactly equivalent. So, for example, if a scale-up event is triggered by a pod which needs a zone-specific PVC (e.g. an EBS volume), the new node might get scheduled in the wrong AZ and the pod will fail to start.

You won't need a separate nodegroup for each AZ if your environment meets the following criteria:

- No zone-specific storage requirements.
- No required podAffinity with topology other than host.
- No required nodeAffinity on zone label.
- No nodeSelector on a zone label.

(Read more [here](https://github.com/kubernetes/autoscaler/pull/1802#issuecomment-474295002) and [here](https://github.com/weaveworks/eksctl/pull/647#issuecomment-474698054).)

If you meet all of the above requirements (and possibly others) then you should be safe with a single nodegroup which spans multiple AZs. Otherwise you'll want to create separate, single-AZ nodegroups:

BEFORE:

```yaml
nodeGroups:
- name: ng1-public
instanceType: m5.xlarge
# availabilityZones: ["eu-west-2a", "eu-west-2b"]
```

AFTER:

```yaml
nodeGroups:
- name: ng1-public-2a
instanceType: m5.xlarge
availabilityZones: ["eu-west-2a"]
- name: ng1-public-2b
instanceType: m5.xlarge
availabilityZones: ["eu-west-2b"]
```

### VPC Networking

By default, `eksctl create cluster` will build a dedicated VPC, in order to avoid interference with any existing resources for a
Expand Down
1 change: 1 addition & 0 deletions humans.txt
Expand Up @@ -36,6 +36,7 @@ Yandy Ramirez @IPyandy
Jerry Jackson @jrryjcksn
Dann Church @D3nn
Roli Schilter @rndstr
Mitchel Humpherys @mgalgs

/* Thanks */

Expand Down