[proposal] A more flexible, scalable and reliable descheduler

[eahydra]

What is your proposal:

We should enhance the k8s descheduler to make it be more flexible, scalable and reliable, and make the Koordinator's scheduling capabilities more complete.

Why is this needed:

• We should extend the descheduler to support event-triggered scenarios. The current k8s descheduler is periodic interval, but many scenarios are triggered based on events.
• We should try to ensure that the descheduled Pod also has resources to allocate. The current k8s descheduler only evict abnormal Pods, but don't care about resources.
• We need a more powerful eviction API, not a simple Eviction. A more powerful eviction API not only needs to evict Pods more safely and reliably, but also helps us complete some complex processes. For example, we can arbitrate descheduling decisions initiated by different scenarios and select more suitable Pods for eviction; Give control to users or higher-level systems to help decision-making; or strengthen auditing mechanisms to trace who evicts a Pod for what reason, and what the final result is.
• Support more co-location scenarios like load-aware descheduling, defragmentation, orchestrating SLO, application-metrics descheduling, etc..

Is there a suggested solution, if so, please add it:

• Submit a proposal for detailed design about descheduler #199
  • [proposal] design the new eviction api  #357
• Implementation
  • implementation the descheduler framework to support the event-triggered scenarios, and compatible with community frameworks #213
  • implementation the eviction api #214
  • implementation some descheduling plugins

[eahydra]

the descheduler relies on the reserveResource mechanism. #188

[eahydra]

/area koord-descheduler