Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(autoscaling): every deployment resets capacity #5507

Merged
merged 5 commits into from
Dec 23, 2019

Conversation

rix0rrr
Copy link
Contributor

@rix0rrr rix0rrr commented Dec 20, 2019

If DesiredCapacity is specified in the CloudFormation template, on every
deployment the capacity of the AutoScalingGroup is reset to that number,
even if the group had been scaled out at that point. The solution is to
leave DesiredCapacity empty, in which case it will remain untouched
during a deployment.

Previously, CDK would use some logic to always calculate a
DesiredCapacity for you, even if you left the desiredCapacity property
unset, leading to the undesirable behavior--which frankly represents
an availability risk.

Now, if you don't specify desiredCapacity, we won't set
DesiredCapacity either, avoiding the availability risk that we
introduced beforehand. In fact, if you do set desiredCapacity, we
will warn you that you probably shouldn't using a construct warning.

Fixes #5215, closes #5208.

BREAKING CHANGE: AutoScalingGroups without desiredCapacity are now
initially scaled to their minimum capacity (instead of their maximum
capaciety).


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

If `DesiredCapacity` is specified in the CloudFormation template, on every
deployment the capacity of the AutoScalingGroup is reset to that number,
even if the group had been scaled out at that point. The solution is to
leave DesiredCapacity empty, in which case it will remain untouched
during a deployment.

Previously, CDK would use some logic to always calculate a
DesiredCapacity for you, even if you left the `desiredCapacity` property
unset, leading to the undesirable behavior--which frankly represents
an availability risk.

Now, if you don't specify `desiredCapacity`, we won't set
`DesiredCapacity` either, avoiding the availability risk that we
introduced beforehand. In fact, if you *do* set `desiredCapacity`, we
will warn you that you probably shouldn't using a construct warning.

Fixes #5215, closes #5208.

BREAKING CHANGE: AutoScalingGroups without `desiredCapacity` are now
initially scaled to their minimum capacity (instead of their maximum
capaciety).
@rix0rrr rix0rrr self-assigned this Dec 20, 2019
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Dec 20, 2019
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

* If this is set to a number, every deployment will reset the amount of
* instances to this number. It is recommended to leave this value blank.
*
* @default minCapacity, and leave unchanged during deployment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this accurate, it doesn't look like you're defaulting desiredCapacity

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This describes the end-to-end behavior of what will happen if you leave the value empty.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-group.html#cfn-as-group-desiredcapacity

If you do not specify a desired capacity, the default is the minimum size of the group.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add this link in the docstring

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

* If this is set to a number, every deployment will reset the amount of
* instances to this number. It is recommended to leave this value blank.
*
* @default minCapacity, and leave unchanged during deployment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add this link in the docstring

if (max < desired) {
throw new Error(`Should have desiredCapacity (${desired}) <= maxCapacity (${max})`);
}
});

if (desiredCapacity !== undefined) {
this.node.addWarning(`desiredCapacity has been configured. Be aware this will reset the size of your AutoScalingGroup on every deployment.`);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provider a link to a github issue in the help message so people can dive deeper/comment.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Contributor

mergify bot commented Dec 23, 2019

Thank you for contributing! Your pull request is now being automatically merged.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Contributor

mergify bot commented Dec 23, 2019

Thank you for contributing! Your pull request is now being automatically merged.

@mergify mergify bot merged commit 0adf6c7 into master Dec 23, 2019
@mergify mergify bot deleted the huijbers/fix-desired-capacity branch December 23, 2019 16:26
eladb pushed a commit that referenced this pull request Dec 24, 2019
* fix(autoscaling): every deployment resets capacity

If `DesiredCapacity` is specified in the CloudFormation template, on every
deployment the capacity of the AutoScalingGroup is reset to that number,
even if the group had been scaled out at that point. The solution is to
leave DesiredCapacity empty, in which case it will remain untouched
during a deployment.

Previously, CDK would use some logic to always calculate a
DesiredCapacity for you, even if you left the `desiredCapacity` property
unset, leading to the undesirable behavior--which frankly represents
an availability risk.

Now, if you don't specify `desiredCapacity`, we won't set
`DesiredCapacity` either, avoiding the availability risk that we
introduced beforehand. In fact, if you *do* set `desiredCapacity`, we
will warn you that you probably shouldn't using a construct warning.

Fixes #5215, closes #5208.

BREAKING CHANGE: AutoScalingGroups without `desiredCapacity` are now
initially scaled to their minimum capacity (instead of their maximum
capaciety).

* Add links

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
eladb pushed a commit that referenced this pull request Jan 5, 2020
As described in #5215, `desiredCapacity` is not the recommended way to configure an auto scaling group since it will cause the ASG to reset the number of nodes in every CloudFormation deployment. Since EKS's default capacity uses `desiredCapacity` instead of `minCapacity`, as of #5507 this would emit a warning: "desiredCapacity has been configured. Be aware this will reset the size of your AutoScalingGroup on every deployment".

This change modifies the behavior of the default capacity such that it will configure the ASG using `minCapacity` instead of `desiredCapacity` as recommended by ASG.

Fixes #5650
mergify bot added a commit that referenced this pull request Jan 6, 2020
…ern (#5651)

* fix(eks): default capacity uses desiredCapacity which is an anti-pattern

As described in #5215, `desiredCapacity` is not the recommended way to configure an auto scaling group since it will cause the ASG to reset the number of nodes in every CloudFormation deployment. Since EKS's default capacity uses `desiredCapacity` instead of `minCapacity`, as of #5507 this would emit a warning: "desiredCapacity has been configured. Be aware this will reset the size of your AutoScalingGroup on every deployment".

This change modifies the behavior of the default capacity such that it will configure the ASG using `minCapacity` instead of `desiredCapacity` as recommended by ASG.

Fixes #5650

* Update integ.eks-cluster.defaults.expected.json

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Preserve current desired counts on deploy of auto scaled resources
4 participants