Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate instance type before CloudFormation #1007

Merged
merged 5 commits into from May 19, 2020
Merged

Validate instance type before CloudFormation #1007

merged 5 commits into from May 19, 2020

Conversation

sonofachamp
Copy link
Contributor

@sonofachamp sonofachamp commented Apr 8, 2020

CloudFormation will try to validate the Default ECSInstanceType parameter
based on the AllowedValues field which is problematic in regions that do
not support our default t2.micro instance type such as eu-north-1.

Resolves #1006


Enter [N/A] in the box, if an item is not applicable to your change.

Testing

  • Unit tests passed
  • Integration tests passed
  • Unit tests added for new functionality
  • Listed manual checks and their outputs in the comments (example)
  • Link to issue or PR for the integration tests:

Documentation

  • Contacted our doc writer
  • Updated our README

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

CloudFormation will try to validate the Default ECSInstanceType parameter
based on the AllowedValues field which is problematic in regions that do
not support our default t2.micro instance type such as eu-north-1.

Resolves #1006
@sonofachamp sonofachamp requested review from iamhopaul123 and a team April 8, 2020 18:25
@sonofachamp sonofachamp self-assigned this Apr 8, 2020
Copy link
Contributor

@SoManyHs SoManyHs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thank you!! Can you add manual test output in this PR too? Thanks!

ecs-cli/modules/cli/cluster/cluster_app.go Outdated Show resolved Hide resolved
ecs-cli/modules/cli/cluster/cluster_app.go Show resolved Hide resolved
Copy link
Contributor

@iamhopaul123 iamhopaul123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM just nits

ecs-cli/modules/cli/cluster/cluster_app.go Show resolved Hide resolved
ecs-cli/modules/cli/cluster/cluster_app.go Outdated Show resolved Hide resolved
Comment on lines 422 to 433
param, err := cfnParams.GetParameter(ParameterKeyInstanceType)
if err == cloudformation.ParameterNotFoundError {
logrus.Infof("Defaulting instance type to %s", cloudformation.DefaultECSInstanceType)

cfnParams.Add(ParameterKeyInstanceType, cloudformation.DefaultECSInstanceType)

return cloudformation.DefaultECSInstanceType, nil
} else if err != nil {
return "", err
}

return aws.StringValue(param.ParameterValue), err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
param, err := cfnParams.GetParameter(ParameterKeyInstanceType)
if err == cloudformation.ParameterNotFoundError {
logrus.Infof("Defaulting instance type to %s", cloudformation.DefaultECSInstanceType)
cfnParams.Add(ParameterKeyInstanceType, cloudformation.DefaultECSInstanceType)
return cloudformation.DefaultECSInstanceType, nil
} else if err != nil {
return "", err
}
return aws.StringValue(param.ParameterValue), err
param, err := cfnParams.GetParameter(ParameterKeyInstanceType)
if err == nil {
return aws.StringValue(param.ParameterValue), err
}
if err == cloudformation.ParameterNotFoundError {
logrus.Infof("Defaulting instance type to %s", cloudformation.DefaultECSInstanceType)
cfnParams.Add(ParameterKeyInstanceType, cloudformation.DefaultECSInstanceType)
return cloudformation.DefaultECSInstanceType, nil
}
return "", err

Maybe like this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's more standard to left-align the happy path. We should check for non-nil errors first and handle those.

@sonofachamp
Copy link
Contributor Author

Sample output runs

Unsupported instance type flag

./bin/local/ecs-cli up --region eu-north-1 --capability-iam --launch-type EC2 --instance-type t2.micro

WARN[0000] You will not be able to SSH into your EC2 instances without a key pair. 
WARN[0001] Default instance type t2.micro not supported in region eu-north-1. Override the default instance type with the --instance-type flag and provide a supported value. 
FATA[0001] Error executing 'up': instance type t2.micro not supported in region eu-north-1: instance type t2.micro not found in list of supported instance types [r5d.16xlarge t3.micro i3.4xlarge c5d.18xlarge i3.xlarge m5d.24xlarge g4dn.8xlarge r5.xlarge r5.16xlarge c5.4xlarge c5d.4xlarge g4dn.4xlarge m5.2xlarge m5.metal r5d.2xlarge g4dn.16xlarge c5d.metal m5d.8xlarge m5d.metal m5d.12xlarge i3.large t3.2xlarge d2.2xlarge c5.12xlarge c5d.12xlarge r5.8xlarge t3.nano r5d.xlarge r5d.metal m5d.xlarge c5d.xlarge m5d.2xlarge m5.16xlarge r5d.24xlarge c5.large t3.medium r5.2xlarge c5d.large i3.2xlarge g4dn.2xlarge r5.metal r5d.12xlarge c5d.24xlarge c5.9xlarge g4dn.xlarge m5d.16xlarge r5d.8xlarge r5d.4xlarge r5.large m5.xlarge d2.8xlarge d2.xlarge m5.4xlarge m5.large m5.24xlarge r5d.large r5.4xlarge c5d.2xlarge m5d.large r5.24xlarge t3.small c5.2xlarge i3.16xlarge r5.12xlarge c5.metal i3.8xlarge c5.xlarge m5.12xlarge c5.18xlarge c5d.9xlarge m5.8xlarge t3.xlarge c5.24xlarge t3.large m5d.4xlarge g4dn.12xlarge d2.4xlarge]

Default instance type is unsupported in specified region

./bin/local/ecs-cli up --region eu-north-1 --capability-iam --launch-type EC2

WARN[0000] You will not be able to SSH into your EC2 instances without a key pair. 
INFO[0000] Defaulting instance type to t2.micro         
WARN[0001] Default instance type t2.micro not supported in region eu-north-1. Override the default instance type with the --instance-type flag and provide a supported value. 
FATA[0001] Error executing 'up': instance type t2.micro not supported in region eu-north-1: instance type t2.micro not found in list of supported instance types [r5d.12xlarge d2.xlarge r5d.24xlarge c5.xlarge t3.2xlarge m5.large i3.large g4dn.4xlarge c5d.metal m5d.large m5.24xlarge m5.metal r5.16xlarge t3.large g4dn.2xlarge t3.xlarge i3.4xlarge c5.large m5d.8xlarge r5d.4xlarge c5d.4xlarge m5d.16xlarge t3.nano g4dn.16xlarge r5d.large r5.12xlarge c5.12xlarge r5.metal c5.metal c5.18xlarge i3.xlarge c5d.large c5.2xlarge r5.xlarge g4dn.xlarge m5d.4xlarge m5d.2xlarge t3.medium m5d.metal m5d.12xlarge c5d.2xlarge m5.4xlarge m5.8xlarge m5.12xlarge c5.4xlarge r5.large m5.2xlarge m5d.xlarge i3.16xlarge c5d.9xlarge t3.small r5.8xlarge i3.2xlarge r5d.xlarge r5d.metal m5.xlarge r5d.8xlarge c5d.18xlarge t3.micro c5d.xlarge d2.4xlarge d2.2xlarge g4dn.8xlarge c5d.24xlarge g4dn.12xlarge m5.16xlarge c5.9xlarge r5.24xlarge r5.2xlarge c5.24xlarge m5d.24xlarge r5d.2xlarge r5d.16xlarge c5d.12xlarge r5.4xlarge d2.8xlarge i3.8xlarge]

Instance type flag is supported in specified region

./bin/local/ecs-cli up --region eu-north-1 --capability-iam --launch-type EC2 --instance-type t3.small

WARN[0000] You will not be able to SSH into your EC2 instances without a key pair. 
INFO[0002] Using recommended Amazon Linux 2 AMI with ECS Agent 1.39.0 and Docker version 18.09.9-ce 
INFO[0003] Created cluster                               cluster=pnt-raw region=eu-north-1
INFO[0005] Waiting for your cluster resources to be created... 
INFO[0005] Cloudformation stack status                   stackStatus=CREATE_IN_PROGRESS
INFO[0067] Cloudformation stack status                   stackStatus=CREATE_IN_PROGRESS
INFO[0130] Cloudformation stack status                   stackStatus=CREATE_IN_PROGRESS
VPC created: <redacted>
Security Group created: <redacted>
Subnet created: <redacted>
Subnet created: <redacted>
Cluster creation succeeded.

Copy link
Contributor

@iamhopaul123 iamhopaul123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you!

@sonofachamp
Copy link
Contributor Author

I still need to run the integration tests against these proposed changes.

@sonofachamp
Copy link
Contributor Author

Integ tests are passing now. I had to add Default: "" for the EcsInstanceType parameter for the TestCreateClusterWithFargateService to pass even though this parameter has no significance in the context of Fargate. Ideally this would be an "optional parameter" so CloudFormation doesn't kick it out if we don't provide a value, or we would not provide this parameter in the template for Fargate.

@elbow
Copy link

elbow commented Apr 29, 2020

Would be keen to get this merged.
Currently you can't use ecs-cli up in af-south-1 since this new region doesn't have t2.micro.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Supported instance type being rejected in region eu-north-1
4 participants