Weighted provisioners

[jonathan-innis]

Fixes #2024

Description

• Allows for the ordering of Provisioners based off a .spec.weight which will be used during scheduling

How was this change tested?

• make test

Does this change impact docs?

• Yes, PR includes docs updates
• Yes, issue opened: #
• No

Release Note

None

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[netlify]

✅ Deploy Preview for karpenter-docs-prod ready!

Name	Link
🔨 Latest commit	608d9a2
🔍 Latest deploy log	https://app.netlify.com/sites/karpenter-docs-prod/deploys/62fe7d22a1ea5b0008230291
😎 Deploy Preview	https://deploy-preview-2307--karpenter-docs-prod.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[debugger24]

If there are 2 provisioner

Provisioner 1

• weight: 100
• instance types: large instances

Provisioner 2

• weight: 50
• instance types: medium + large instances

---

Now, if there is requirement to add a single pod for which even small instance size is enough, will it still create resource using provisioner 1 config because of higher weight? or it will use provisoner 2 config since karpenter also tries to create least size instance when possible.

[jonathan-innis]

Now, if there is requirement to add a single pod for which even small instance size is enough, will it still create resource using provisioner 1 config because of higher weight? or it will use provisoner 2 config since karpenter also tries to create least size instance when possible.

In this case, it would choose provisioner 1 regardless if the cost would be cheaper on provisioner 2. As long as the pod can be scheduled to that first provisioner, Karpenter will launch the node with the highest priority.

The way to solve the issue you have described above is to make sure that you have enough flexibility in your instance types in your provisioners such that the scheduler is not forced into a situation like this

[ellistarn]

LGTM. A few nits.

[Adityashar]

Hey @jonathan-innis,

I have a question regarding the usability of these weighted provisioners.

Suppose I have two provisioners -

1. provisioner-spot with weight: 40
2. provisioner-on-demand with weight: 60

then as per the code, since our provisioners are getting sorted with respect to their weights, there may arise a situation where only my provisioner-on-demand is being used for provisioning nodes and the other provisioner becomes stale. Is this how we are intending weighted provisioners to work?

Also, could we work upon changing this logic to something like a percentage based usage of provisioners? Eg. If my provisioner-on-demand has a weight equal to 60, then it means that this provisioner will be used for fulfilling 60% of my node provisioning requests and similarly the other provisioner responds to the remaining 40% of the requests.