-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
pkg/types/aws/defaults/platform: Per-region instance-class defaults
From Trevor: For regions like eu-west-3 (Paris) that don't support m4. The only instance type available in all AWS regions today is i3. We prefer m4 because of the higher EBS attach limit. But for regions that don't support m4 we use m5. This also adds a CI job which checks the AWS pricing API to keep our region map up to date. The eu-north-1 region is new as of 2018-12-12 [1], and it show up in: $ aws ec2 describe-regions --query 'Regions[].RegionName' --output json | jq -r 'sort[]' ... eu-central-1 eu-north-1 eu-west-1 ... The GovCloud entries didn't show up in that query (maybe they are only listed for accounts that have access?), but they are listed on [2]. The new test is in platformtests/aws to keep it out of our default unit tests. Maintainers can run it manually to verify that the default fallbacks we hard-code are still appropriate for the current AWS offerings. It's not in tests/aws, because 'dep ensure' seems to ignore tests. I think it's fine to have both platformtests and tests in parallel, with the former holding tests who share the project vendor/ and the latter holding tests which have a distinct vendor/. bdd-smoke is in tests with its own vendor/ intentionally to keep gomega and other test-specific libraries out of the project vendor/ [3]. But for these tests we're using Go's built-in test framework, so it's less maintenance to layer our minimal additional dependencies into the project vendor/. [1]: https://aws.amazon.com/blogs/aws/now-open-aws-europe-stockholm-region/ [2]: https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ [3]: #552 (comment)
- Loading branch information
Showing
14 changed files
with
202 additions
and
25 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # Platform Tests | ||
|
|
||
| This directory contains test suites checking per-platform assumptions. | ||
| These tests require platform access that is not appropriate for platform-agnostic unit tests. | ||
| The `t` directory name (unlike `tests`) allows us to share [the project `vendor` directory managed by `dep`](../docs/dev/dependencies.md#go). | ||
|
|
||
| Platforms: | ||
|
|
||
| * [aws](aws) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # AWS Tests | ||
|
|
||
| This directory contains test suites checking AWS-specific assumptions. | ||
| Run with: | ||
|
|
||
| ```console | ||
| $ AWS_PROFILE=your-profile go test . | ||
| ``` | ||
|
|
||
| or similar (it needs access to [your AWS credentials][credentials]). | ||
|
|
||
| [credentials]: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,110 @@ | ||
| package aws | ||
|
|
||
| import ( | ||
| "strings" | ||
| "testing" | ||
|
|
||
| "github.com/aws/aws-sdk-go/aws" | ||
| "github.com/aws/aws-sdk-go/service/pricing" | ||
| awsutil "github.com/openshift/installer/pkg/asset/installconfig/aws" | ||
| "github.com/openshift/installer/pkg/types/aws/defaults" | ||
| "github.com/openshift/installer/pkg/types/aws/validation" | ||
| "github.com/stretchr/testify/assert" | ||
| ) | ||
|
|
||
| func TestGetDefaultInstanceClass(t *testing.T) { | ||
| ssn, err := awsutil.GetSession() | ||
| if err != nil { | ||
| t.Fatal(err) | ||
| } | ||
|
|
||
| exists := struct{}{} | ||
| instanceClasses := map[string]map[string]struct{}{} | ||
|
|
||
| client := pricing.New(ssn, aws.NewConfig().WithRegion("us-east-1")) | ||
|
|
||
| err = client.GetProductsPages( | ||
| &pricing.GetProductsInput{ | ||
| ServiceCode: aws.String("AmazonEC2"), | ||
| Filters: []*pricing.Filter{ | ||
| { | ||
| Field: aws.String("tenancy"), | ||
| Type: aws.String("TERM_MATCH"), | ||
| Value: aws.String("Shared"), | ||
| }, | ||
| { | ||
| Field: aws.String("productFamily"), | ||
| Type: aws.String("TERM_MATCH"), | ||
| Value: aws.String("Compute Instance"), | ||
| }, | ||
| { | ||
| Field: aws.String("operatingSystem"), | ||
| Type: aws.String("TERM_MATCH"), | ||
| Value: aws.String("Linux"), | ||
| }, | ||
| { | ||
| Field: aws.String("instanceFamily"), | ||
| Type: aws.String("TERM_MATCH"), | ||
| Value: aws.String("General purpose"), | ||
| }, | ||
| }, | ||
| }, | ||
| func(result *pricing.GetProductsOutput, lastPage bool) bool { | ||
| for _, priceList := range result.PriceList { | ||
| product := priceList["product"].(map[string]interface{}) | ||
| attr := product["attributes"].(map[string]interface{}) | ||
| location := attr["location"].(string) | ||
| instanceType := attr["instanceType"].(string) | ||
| instanceClassSlice := strings.Split(instanceType, ".") | ||
| instanceClass := instanceClassSlice[0] | ||
| _, ok := instanceClasses[location] | ||
| if ok { | ||
| instanceClasses[location][instanceClass] = exists | ||
| } else { | ||
| instanceClasses[location] = map[string]struct{}{instanceClass: exists} | ||
| } | ||
| } | ||
| return !lastPage | ||
| }, | ||
| ) | ||
| if err != nil { | ||
| t.Fatal(err) | ||
| } | ||
|
|
||
| regions := map[string]string{ // seed with locations that don't match AWS's usual names | ||
| "South America (Sao Paulo)": "sa-east-1", | ||
| "AWS GovCloud (US)": "us-gov-west-1", | ||
| } | ||
|
|
||
| for location, classes := range instanceClasses { | ||
| t.Run(location, func(t *testing.T) { | ||
| region, ok := regions[location] | ||
| if !ok { | ||
| for slug, name := range validation.Regions { | ||
| if strings.Contains(location, name) { | ||
| regions[location] = slug | ||
| region = slug | ||
| break | ||
| } | ||
| } | ||
| if region == "" { | ||
| t.Fatal("not a recognized region") | ||
| } | ||
| } | ||
|
|
||
| class := "" | ||
| // ordered list of prefered instance classes | ||
| for _, preferredClass := range []string{"m4", "m5"} { | ||
| if _, ok := classes[preferredClass]; ok { | ||
| class = preferredClass | ||
| break | ||
| } | ||
| } | ||
| if class == "" { | ||
| t.Fatalf("does not support any preferred classes: %v", classes) | ||
| } | ||
| defaultClass := defaults.InstanceClass(region) | ||
| assert.Equal(t, defaultClass, class) | ||
| }) | ||
| } | ||
| } |