KEP-693: MultiKueue #1380
KEP-693: MultiKueue #1380
Conversation
k8s-ci-robot
added
release-note-none
✅ Deploy Preview for kubernetes-sigs-kueue canceled.
|
k8s-ci-robot
added
the
size/L
|
cc: @trasc |
|
cc @dejanzele |
keps/693-multikueue/README.md
Outdated
| Then it will remove the workloads from the remaining clusters and allow the | ||
| single instance of the job to proceed. The workload will be also admitted in | ||
| the management cluster. |
There was a problem hiding this comment.
| Then it will remove the workloads from the remaining clusters and allow the | |
| single instance of the job to proceed. The workload will be also admitted in | |
| the management cluster. | |
| Then it will remove the workloads from the remaining remote clusters and allow the | |
| single instance of the job to proceed. The local workload will get the admission check set to retry in order to free the local quota. |
There was a problem hiding this comment.
Why to set the setting to retry? There is no local quota in the managment cluster.
|
How will you handle if a worker cluster fails for some reason (network issues, cluster goes down...)? Will you have some sort of job leases and job periodically reporting back they are still executing? |
k8s-ci-robot
added
the
approved
|
Thanks |
k8s-ci-robot
added
the
do-not-merge/hold
k8s-ci-robot
added
the
lgtm
|
LGTM label has been added. Git tree hash: 62b43f0304ec7815182c32fcebbfecfd87bdb00e
|
| When the job is running MultiKueue controller will copy its status from worker cluster | ||
| to the management cluster, to keep the impression that the job is running in the management | ||
| cluster. This is needed to allow pipelines and workflow engines to execute against | ||
| the management cluster with MultiKueue without any extra changes. |
There was a problem hiding this comment.
What happens when the kueue manager loses connectivity to the worker cluster after some workloads are admitted?
Especially preemption and waitForPodsReady, what happens?
There was a problem hiding this comment.
We assume the total loss of the cluster and all admitted workloads are suspended/requeued. Once the cluster is reconnected, we remove duplicated admitted workloads just as if two of them were admitted at the same time.
Added to the doc.
There was a problem hiding this comment.
If preemption targets exist in the connection loss cluster, what happens?
Kueue scheduler will try to preempt the targets forever, right?
There was a problem hiding this comment.
If preemption targets exist in the connection loss cluster, what happens?
Kueue scheduler will try to preempt the targets forever, right?
I read an updated doc and then I understand what happens in the above situation.
k8s-ci-robot
removed
the
lgtm
|
@tenzen-y Yes, such configuration will be possible in the future, once we establish kubernetes/enhancements#4370 as a universal standard for selectively disabling controllers for other API/CRD objects. Right now the only option for CRDs in the management cluster is to install API definitions but without controllers, that prevents allowing two roles inside one cluster. |
@mwielgus It makes sense. Can we mention that in |
|
@tenzen-y Added. |
k8s-ci-robot
added
the
lgtm
|
LGTM label has been added. Git tree hash: 828418fd930219441c5d1295b67184cce9512373
|
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: alculquicondor, mwielgus, tenzen-y The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/hold cancel |
k8s-ci-robot
removed
the
do-not-merge/hold
What type of PR is this?
/kind feature
What this PR does / why we need it:
Introduce MultiCluster support in Kueue.
Which issue(s) this PR fixes:
Fixes #693
Special notes for your reviewer:
Does this PR introduce a user-facing change?