Skip to content

chore: set tasks fewer than spot_from to fargate cp when min and spot_from equal #4187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mergify merged 5 commits into from
Nov 17, 2022
Merged

chore: set tasks fewer than spot_from to fargate cp when min and spot_from equal #4187

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e47bb01
chore: honor spot_from value if equal to min
huanjani Nov 15, 2022
bc064c8
chore: lowercase capacity provider
huanjani Nov 15, 2022
9bcbede
chore: add validation to ensure positive range values
huanjani Nov 17, 2022
720f9ec
chore: make negative base values zero
huanjani Nov 17, 2022
43d1511
Merge branch 'mainline' into spot
mergify[bot] Nov 17, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions internal/pkg/deploy/cloudformation/stack/transformers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,18 +198,21 @@ func convertCapacityProviders(a manifest.AdvancedCount) []*template.CapacityProv
CapacityProvider: capacityProviderFargateSpot,
})
rc := a.Range.RangeConfig
// Return if only spot is specifed as count
// Return if only spot is specified as count
if rc.SpotFrom == nil {
return cps
}
// Scaling with spot
spotFrom := aws.IntValue(rc.SpotFrom)
min := aws.IntValue(rc.Min)
// If spotFrom value is not equal to the autoscaling min, then
// the base value on the Fargate Capacity provider must be set
// If spotFrom value is greater than or equal to the autoscaling min,
// then the base value on the Fargate capacity provider must be set
// to one less than spotFrom
if spotFrom > min {
if spotFrom >= min {
base := spotFrom - 1
Copy link
Copy Markdown
Contributor

@efekarakus efekarakus Nov 16, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

huh so you can you can have spotFrom and min be 0?
and it doesn't error if the base ends up being -1?

According to the commit message, we get:

min: 0 and spot_from: 0 --> 0 Fargate, 1 Spot

I think this is pretty strange, I would have expected 0 Fargate and 0 Spot here.

edit: I take this back 1 fargate spot makes sense if your cpu_percentage is like 70% because when you have 0 tasks the avg cpu is at 0%, so ECS has to scale up resulting in 1 spot task.

Copy link
Copy Markdown
Contributor

@efekarakus efekarakus Nov 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side-note: should we make sure that min, max and spot_from is >= 0 here:

copilot-cli/internal/pkg/manifest/validate.go

Line 1077 in 5781c53

func (r RangeConfig) validate() error {

Copy link
Copy Markdown
Contributor Author

@huanjani huanjani Nov 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screen Shot 2022-11-16 at 3 33 02 PM

Copy link
Copy Markdown
Contributor

@efekarakus efekarakus Nov 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this has got to be a bug lol 😂 like what could -1 base mean conceptually

if base < 0 {
base = 0
}
fgCapacity := &template.CapacityProviderStrategy{
Base: aws.Int(base),
Weight: aws.Int(0),
Expand Down
28 changes: 28 additions & 0 deletions internal/pkg/deploy/cloudformation/stack/transformers_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,11 @@ func Test_convertCapacityProviders(t *testing.T) {
Weight: aws.Int(1),
CapacityProvider: capacityProviderFargateSpot,
},
{
Base: aws.Int(0),
Weight: aws.Int(0),
CapacityProvider: capacityProviderFargate,
},
},
},
"with scaling into spot": {
Expand All @@ -429,6 +434,29 @@ func Test_convertCapacityProviders(t *testing.T) {
},
},
},
"with min equaling spot_from": {
input: manifest.AdvancedCount{
Range: manifest.Range{
RangeConfig: manifest.RangeConfig{
Min: aws.Int(2),
Max: aws.Int(10),
SpotFrom: aws.Int(2),
},
},
},

expected: []*template.CapacityProviderStrategy{
{
Weight: aws.Int(1),
CapacityProvider: capacityProviderFargateSpot,
},
{
Base: aws.Int(1),
Weight: aws.Int(0),
CapacityProvider: capacityProviderFargate,
},
},
},
"returns nil if no spot config specified": {
input: manifest.AdvancedCount{
Range: manifest.Range{
Expand Down
10 changes: 10 additions & 0 deletions internal/pkg/manifest/errors.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,16 @@ http:
`
}

type errRangeValueLessThanZero struct {
min int
max int
spotFrom int
}

func (e *errRangeValueLessThanZero) Error() string{
return fmt.Sprintf("min value %d, max value %d, and spot_from value %d must all be positive", e.min, e.max, e.spotFrom)
}

type errMinGreaterThanMax struct {
min int
max int
Expand Down
9 changes: 8 additions & 1 deletion internal/pkg/manifest/validate.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1080,7 +1080,14 @@ func (r RangeConfig) validate() error {
missingField: "min/max",
}
}
min, max := aws.IntValue(r.Min), aws.IntValue(r.Max)
min, max, spotFrom := aws.IntValue(r.Min), aws.IntValue(r.Max), aws.IntValue(r.SpotFrom)
if min < 0 || max < 0 || spotFrom < 0 {
return &errRangeValueLessThanZero{
min: min,
max: max,
spotFrom: spotFrom,
}
}
if min <= max {
return nil
}
Expand Down
8 changes: 8 additions & 0 deletions internal/pkg/manifest/validate_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2105,6 +2105,14 @@ func TestRangeConfig_validate(t *testing.T) {
},
wantedError: fmt.Errorf("min value 2 cannot be greater than max value 1"),
},
"error if spot_from value is negative": {
RangeConfig: RangeConfig{
Min: aws.Int(2),
Max: aws.Int(10),
SpotFrom: aws.Int(-3),
},
wantedError: fmt.Errorf("min value 2, max value 10, and spot_from value -3 must all be positive"),
},
}
for name, tc := range testCases {
t.Run(name, func(t *testing.T) {
Expand Down