Workload-aware preemption

[wojtek-t]

Enhancement Description

• One-line enhancement description (can be used as a release note): Support workload-aware preemption in Kubernetes scheduler

• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/5710-workload-aware-preemption/README.md

• Primary contact (assignee): @wojtek-t

• Responsible SIGs: sig-scheduling

• Enhancement target (which target equals to which milestone):

  • Alpha release target (x.y): 1.36
  • Beta release target (x.y): 1.37
  • Stable release target (x.y): 1.39

• PRs by stage and milestone:

  • Alpha - v1.36
    • KEP (k/enhancements) update PR(s): KEP-5710: Add initial KEP docs for workload-aware preemption #5711 & KEP-5710: Update KEP to use decoupled PodGroup API. #5919
    • Code (k/k) update PR(s):
      • KEP-5710: Add WorkloadAwarePreemption feature gate kubernetes#137200
      • Add workload aware preemption kubernetes#137606
      • Add Workload-Aware Preemption fields to Workload and PodGroup APIs kubernetes#136589
    • Docs (k/website) update PR(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[rst0git]

@wojtek-t We are also exploring workload-aware preemption for AI training workloads as part of the Checkpoint/Restore working group and KubeFlow Trainer: kubeflow/trainer#2245 kubeflow/trainer#2777

The following papers describe some of our previous work on this topic:

• Kubernetes Scheduling with Checkpoint/Restore: Challenges and Open Problems
• CRIUgpu: Transparent Checkpointing of GPU-Accelerated Workloads

[wojtek-t]

Thanks - will take a look. Also please take a look into the first draft of the KEP and I'm happy to hear/accomodate any feedback.

[rst0git]

The following talk also provides more information on how this checkpointing mechanism has been used with Borg:

• Task Migration at Scale Using CRIU (slides, video recording)

[macsko]

/milestone v1.36
/label lead-opted-in