feat: switch from the spot capacity-optimized-prioritized allocation strategy to price-capacity-optimized #2835

bwagner5 · 2022-11-10T23:38:50Z

Fixes #

Description

Replace Karpenter's default spot allocation strategy from capacity-optimized-prioritized where we prioritized based on pricing data to a purpose-built allocation strategy that does a capacity-pricing trade-off by default on the EC2 Fleet side, called price-capacity-optimized.
https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-ec2-price-capacity-optimized-allocation-strategy-provisioning-ec2-spot-instances/

How was this change tested?

make test
Deployed to cluster and tested some allocations (details below)

Does this change impact docs?

Yes, PR includes docs updates
Yes, issue opened: #
No

Release Note

Karpenter now supports the more cost effective spot allocation strategy, price-capacity-optimized.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Pricing Capacity Optimized Spot Allocation Strategy

Provisioner default

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: default
spec:
  providerRef:
    name: default
  requirements:
  - key: karpenter.k8s.aws/instance-category
    operator: In
    values:
    - c
    - m
    - r
  - key: karpenter.k8s.aws/instance-generation
    operator: Gt
    values:
    - "2"
  - key: karpenter.sh/capacity-type
    operator: In
    values:
    - spot
  - key: kubernetes.io/arch
    operator: In
    values:
    - amd64
  ttlSecondsAfterEmpty: 20

Inflate Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: inflate
  namespace: default
spec:
  progressDeadlineSeconds: 600
  replicas: 20
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: inflate
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: inflate
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: inflate
            topologyKey: kubernetes.io/hostname
      containers:
      - image: public.ecr.aws/eks-distro/kubernetes/pause:3.2
        imagePullPolicy: IfNotPresent
        name: inflate
        resources:
          requests:
            cpu: "1"
            memory: 256M
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30

Capacity Optimized Prioritized Spot Allocation Strategy

kubectl scale deploy/inflate --replicas=10

karpenter-7df78fc977-2g24l controller 2022-11-04T15:04:33.364Z  INFO  controller.provisioning.cloudprovider Launched instance: i-02be01599dc9a04aa, hostname: ip-192-168-83-86.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:04:33.369Z  INFO  controller.provisioning.cloudprovider Launched instance: i-00ea539a6ea800bdd, hostname: ip-192-168-94-129.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot  {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:04:33.372Z  INFO  controller.provisioning.cloudprovider Launched instance: i-028f17e316d9ebec0, hostname: ip-192-168-92-153.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot  {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:04:33.403Z  INFO  controller.provisioning.cloudprovider Launched instance: i-01c388b17baa46fb8, hostname: ip-192-168-92-26.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:04:33.404Z  INFO  controller.provisioning.cloudprovider Launched instance: i-09465af6da871cfa8, hostname: ip-192-168-87-96.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:04:33.405Z  INFO  controller.provisioning.cloudprovider Launched instance: i-035700eb196bef197, hostname: ip-192-168-82-249.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot  {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:04:33.406Z  INFO  controller.provisioning.cloudprovider Launched instance: i-08761f027bcacb7a5, hostname: ip-192-168-76-130.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot  {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:04:33.407Z  INFO  controller.provisioning.cloudprovider Launched instance: i-0a9c9b6efcd0ef8da, hostname: ip-192-168-87-24.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot {"commit": "c363e44-dirty", "provisioner": "default"}

cost-per-30-days=$112.50

kubectl scale deploy/inflate --replicas=20

karpenter-7df78fc977-2g24l controller 2022-11-04T15:05:38.811Z  INFO  controller.provisioning.cloudprovider Launched instance: i-09e5fe68db39c4165, hostname: ip-192-168-186-209.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:05:38.815Z  INFO  controller.provisioning.cloudprovider Launched instance: i-0180062908f0c7a49, hostname: ip-192-168-189-30.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot  {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:05:38.821Z  INFO  controller.provisioning.cloudprovider Launched instance: i-0a771376280d6d748, hostname: ip-192-168-183-173.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:05:38.822Z  INFO  controller.provisioning.cloudprovider Launched instance: i-04aeb15f9419ee995, hostname: ip-192-168-188-96.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot  {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:05:38.851Z  INFO  controller.provisioning.cloudprovider Launched instance: i-04c8461e3e5d3e2e5, hostname: ip-192-168-169-135.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:05:38.855Z  INFO  controller.provisioning.cloudprovider Launched instance: i-0a91b43c7b1106f03, hostname: ip-192-168-190-224.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:05:38.857Z  INFO  controller.provisioning.cloudprovider Launched instance: i-0fcd5d5171341dc3b, hostname: ip-192-168-167-107.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:05:38.857Z  INFO  controller.provisioning.cloudprovider Launched instance: i-05a420043d582dae6, hostname: ip-192-168-160-35.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot  {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:05:38.859Z  INFO  controller.provisioning.cloudprovider Launched instance: i-026d362f1ae3d53eb, hostname: ip-192-168-170-193.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot {"commit": "c363e44-dirty", "provisioner": "default"}
karpenter-7df78fc977-2g24l controller 2022-11-04T15:05:38.950Z  INFO  controller.provisioning.cloudprovider Launched instance: i-0dd1ef304fab5bd70, hostname: ip-192-168-165-64.us-east-2.compute.internal, type: m5.large, zone: us-east-2a, capacityType: spot  {"commit": "c363e44-dirty", "provisioner": "default"}

cost-per-30-days=$140.70

Now let's change to use price-capacity-optimized

kubectl scale deploy/inflate --replicas=10

karpenter-547b9b8478-tmqk2 controller 2022-11-04T14:01:33.204Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0c1730214a597b9af, hostname: ip-192-168-121-244.us-east-2.compute.internal, type: c5n.large, zone: us-east-2b, capacityType: spot	{"commit": "33fc137-dirty", "provisioner": "default"}
karpenter-547b9b8478-tmqk2 controller 2022-11-04T14:01:33.207Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0878ebd587cb4bba0, hostname: ip-192-168-156-185.us-east-2.compute.internal, type: c5d.large, zone: us-east-2c, capacityType: spot	{"commit": "33fc137-dirty", "provisioner": "default"}
karpenter-547b9b8478-tmqk2 controller 2022-11-04T14:01:33.210Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0527ad3d35853c4e8, hostname: ip-192-168-92-131.us-east-2.compute.internal, type: c5d.large, zone: us-east-2a, capacityType: spot	{"commit": "33fc137-dirty", "provisioner": "default"}
karpenter-547b9b8478-tmqk2 controller 2022-11-04T14:01:33.212Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0dbe7b6971bc89b2c, hostname: ip-192-168-101-174.us-east-2.compute.internal, type: c4.large, zone: us-east-2b, capacityType: spot	{"commit": "33fc137-dirty", "provisioner": "default"}
karpenter-547b9b8478-tmqk2 controller 2022-11-04T14:01:33.214Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-09dad5d6838ac5512, hostname: ip-192-168-98-7.us-east-2.compute.internal, type: c5a.large, zone: us-east-2b, capacityType: spot	{"commit": "33fc137-dirty", "provisioner": "default"}
karpenter-547b9b8478-tmqk2 controller 2022-11-04T14:01:33.243Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0327270608e866dbb, hostname: ip-192-168-112-34.us-east-2.compute.internal, type: c5d.large, zone: us-east-2b, capacityType: spot	{"commit": "33fc137-dirty", "provisioner": "default"}
karpenter-547b9b8478-tmqk2 controller 2022-11-04T14:01:33.245Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-085c29c6d02454472, hostname: ip-192-168-134-153.us-east-2.compute.internal, type: c4.large, zone: us-east-2c, capacityType: spot	{"commit": "33fc137-dirty", "provisioner": "default"}
karpenter-547b9b8478-tmqk2 controller 2022-11-04T14:01:34.294Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0c93faf86390554de, hostname: ip-192-168-66-255.us-east-2.compute.internal, type: c5a.large, zone: us-east-2a, capacityType: spot	{"commit": "33fc137-dirty", "provisioner": "default"}

cost-per-30-days=$108.30

kubectl scale deploy/inflate --replicas=20

karpenter-67ffd687fb-nnc2k controller 2022-11-04T14:39:40.040Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-031d681869bf9ce9f, hostname: ip-192-168-58-124.us-east-2.compute.internal, type: c4.large, zone: us-east-2c, capacityType: spot	{"commit": "0b122ce-dirty", "provisioner": "default"}
karpenter-67ffd687fb-nnc2k controller 2022-11-04T14:39:40.043Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0f95afb517a3dfdd1, hostname: ip-192-168-26-88.us-east-2.compute.internal, type: c4.large, zone: us-east-2b, capacityType: spot	{"commit": "0b122ce-dirty", "provisioner": "default"}
karpenter-67ffd687fb-nnc2k controller 2022-11-04T14:39:40.044Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0c7bff1241254b3c4, hostname: ip-192-168-13-231.us-east-2.compute.internal, type: c5n.large, zone: us-east-2b, capacityType: spot	{"commit": "0b122ce-dirty", "provisioner": "default"}
karpenter-67ffd687fb-nnc2k controller 2022-11-04T14:39:40.078Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0a587e95ba49a54a6, hostname: ip-192-168-54-235.us-east-2.compute.internal, type: c5d.large, zone: us-east-2c, capacityType: spot	{"commit": "0b122ce-dirty", "provisioner": "default"}
karpenter-67ffd687fb-nnc2k controller 2022-11-04T14:39:40.102Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-00c9fa6a3f814c902, hostname: ip-192-168-9-160.us-east-2.compute.internal, type: m4.large, zone: us-east-2b, capacityType: spot	{"commit": "0b122ce-dirty", "provisioner": "default"}
karpenter-67ffd687fb-nnc2k controller 2022-11-04T14:39:40.103Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0d383033337910752, hostname: ip-192-168-39-234.us-east-2.compute.internal, type: c5n.large, zone: us-east-2c, capacityType: spot	{"commit": "0b122ce-dirty", "provisioner": "default"}
karpenter-67ffd687fb-nnc2k controller 2022-11-04T14:39:40.104Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-05ca8b04a0bdff6c5, hostname: ip-192-168-176-114.us-east-2.compute.internal, type: c5d.large, zone: us-east-2a, capacityType: spot	{"commit": "0b122ce-dirty", "provisioner": "default"}
karpenter-67ffd687fb-nnc2k controller 2022-11-04T14:39:40.106Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-062eddf67cef69814, hostname: ip-192-168-6-239.us-east-2.compute.internal, type: c5a.large, zone: us-east-2b, capacityType: spot	{"commit": "0b122ce-dirty", "provisioner": "default"}
karpenter-67ffd687fb-nnc2k controller 2022-11-04T14:39:40.107Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0a727317f6479f26b, hostname: ip-192-168-17-74.us-east-2.compute.internal, type: c5d.large, zone: us-east-2b, capacityType: spot	{"commit": "0b122ce-dirty", "provisioner": "default"}
karpenter-67ffd687fb-nnc2k controller 2022-11-04T14:39:40.241Z	INFO	controller.provisioning.cloudprovider	Launched instance: i-0873ac36de293c8c3, hostname: ip-192-168-180-124.us-east-2.compute.internal, type: c5a.large, zone: us-east-2a, capacityType: spot	{"commit": "0b122ce-dirty", "provisioner": "default"}

cost-per-30-days=$136.20

This equates to about a 3% discount, in this fairly contrived example, for basically the same capacity and chance of interruption.

Pricing

> ec2-instance-selector --allow-list '(c5.large|c5n.large|c5d.large|m5.large|c4.large|c5a.large)' -o table-wide --region us-east-2
Instance Type  VCPUs   Mem (GiB)  Hypervisor  Current Gen  Hibernation Support  CPU Arch  Network Performance  ENIs    GPUs    GPU Mem (GiB)  GPU Info  On-Demand Price/Hr  Spot Price/Hr (30d avg)
-------------  -----   ---------  ----------  -----------  -------------------  --------  -------------------  ----    ----    -------------  --------  ------------------  -----------------------
c4.large       2       3.75       xen         true         true                 x86_64    Moderate             3       0       0              none      $0.1                $0.01831
c5.large       2       4          nitro       true         true                 x86_64    Up to 10 Gigabit     3       0       0              none      $0.085              $0.01955
c5a.large      2       4          nitro       true         false                x86_64    Up to 10 Gigabit     3       0       0              none      $0.077              $0.019
c5d.large      2       4          nitro       true         true                 x86_64    Up to 10 Gigabit     3       0       0              none      $0.096              $0.019
c5n.large      2       5.25       nitro       true         false                x86_64    Up to 25 Gigabit     3       0       0              none      $0.108              $0.019
m4.large       2       8          xen         true         true                 x86_64    Moderate             2       0       0              none      $0.1                $0.01962
m5.large       2       8          nitro       true         true                 x86_64    Up to 10 Gigabit     3       0       0              none      $0.096              $0.02013

netlify · 2022-11-10T23:38:56Z

✅ Deploy Preview for karpenter-docs-prod canceled.

Name	Link
🔨 Latest commit	`cd85eef`
🔍 Latest deploy log	https://app.netlify.com/sites/karpenter-docs-prod/deploys/636dd2c699485800080d6345

website/content/en/preview/faq.md

…strategy to price-capacity-optimized

Co-authored-by: Jonathan Innis <jonathan.innis.ji@gmail.com>

jonathan-innis

LGTM 🚀

munjalpatel · 2022-11-12T21:40:24Z

This is awesome! Will it be part of the next release? When is the next release?

FernandoMiguel · 2022-11-13T16:51:56Z

It's now in the blog https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-ec2-price-capacity-optimized-allocation-strategy-provisioning-ec2-spot-instances/

malikdraz · 2023-02-02T06:36:35Z

Is there any update on this? Is it a default behaviour for Karpenter now? or can we change this behaviour through Provisioner?

jonathan-innis · 2023-02-02T18:21:23Z

@malikdraz This is the default strategy for the latest versions of Karpenter.

bwagner5 requested a review from a team as a code owner November 10, 2022 23:38

bwagner5 requested a review from spring1843 November 10, 2022 23:38

jonathan-innis reviewed Nov 11, 2022

View reviewed changes

website/content/en/preview/faq.md Outdated Show resolved Hide resolved

website/content/en/preview/faq.md Outdated Show resolved Hide resolved

ellistarn previously approved these changes Nov 11, 2022

View reviewed changes

bwagner5 dismissed ellistarn’s stale review via daf8d1b November 11, 2022 01:52

bwagner5 enabled auto-merge (squash) November 11, 2022 02:59

bwagner5 and others added 3 commits November 10, 2022 22:22

feat: switch from the spot capacity-optimized-prioritized allocation …

c4df51a

…strategy to price-capacity-optimized

Update website/content/en/preview/faq.md

5bb754d

Co-authored-by: Jonathan Innis <jonathan.innis.ji@gmail.com>

Update website/content/en/preview/faq.md

cf02998

Co-authored-by: Jonathan Innis <jonathan.innis.ji@gmail.com>

bwagner5 force-pushed the pco branch from 19020cf to cf02998 Compare November 11, 2022 04:22

cleanup

cd85eef

bwagner5 force-pushed the pco branch from 4ebed8a to cd85eef Compare November 11, 2022 04:42

bwagner5 requested a review from jonathan-innis November 11, 2022 16:23

jonathan-innis approved these changes Nov 11, 2022

View reviewed changes

bwagner5 merged commit a8e8759 into aws:main Nov 11, 2022

bwagner5 deleted the pco branch November 11, 2022 17:38

ellistarn mentioned this pull request Nov 30, 2022

instance size preference - why it chooses the biggest ec2 when only 1 pod is requested #2942

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: switch from the spot capacity-optimized-prioritized allocation strategy to price-capacity-optimized #2835

feat: switch from the spot capacity-optimized-prioritized allocation strategy to price-capacity-optimized #2835

bwagner5 commented Nov 10, 2022 •

edited

netlify bot commented Nov 10, 2022 •

edited

jonathan-innis left a comment

munjalpatel commented Nov 12, 2022

FernandoMiguel commented Nov 13, 2022

malikdraz commented Feb 2, 2023

jonathan-innis commented Feb 2, 2023

feat: switch from the spot capacity-optimized-prioritized allocation strategy to price-capacity-optimized #2835

feat: switch from the spot capacity-optimized-prioritized allocation strategy to price-capacity-optimized #2835

Conversation

bwagner5 commented Nov 10, 2022 • edited

Pricing Capacity Optimized Spot Allocation Strategy

Provisioner default

Inflate Deployment

Capacity Optimized Prioritized Spot Allocation Strategy

kubectl scale deploy/inflate --replicas=10

kubectl scale deploy/inflate --replicas=20

Now let's change to use price-capacity-optimized

kubectl scale deploy/inflate --replicas=10

kubectl scale deploy/inflate --replicas=20

Pricing

netlify bot commented Nov 10, 2022 • edited

✅ Deploy Preview for karpenter-docs-prod canceled.

jonathan-innis left a comment

Choose a reason for hiding this comment

munjalpatel commented Nov 12, 2022

FernandoMiguel commented Nov 13, 2022

malikdraz commented Feb 2, 2023

jonathan-innis commented Feb 2, 2023

bwagner5 commented Nov 10, 2022 •

edited

netlify bot commented Nov 10, 2022 •

edited