koord-descheduler: support soft eviction takeover eviction behavior

[leason00]

Ⅰ. Describe what this PR does

By tagging, the user takes over the eviction.

Ⅱ. Does this pull request fix one issue?

fixes #1102

Ⅲ. Describe how to verify it

1. Enable this feature in koord-descheduler-config

pluginConfig:
- name: MigrationController
  args:
    apiVersion: descheduler/v1alpha2
    kind: MigrationControllerArgs
    evictionPolicy: SoftEviction
    evictLocalStoragePods: true
    ignorePvcPods: true
    namespaces:
      include:
          - qa
    evictQPS: "10"
    evictBurst: 1

2. A annotation will be written on Pod

apiVersion: v1
kind: Pod
metadata:
  annotations:
   scheduling.koordinator.sh/soft-eviction: |-
          {
            "timestamp": 1679970640,
            "deleteOptions": {},
            "initiator": "xxx",
            "reason": "xxx"
          }

Ⅳ. Special notes for reviews

V. Checklist

• I have written necessary docs and comments
• I have added necessary unit tests and integration tests
• All checks passed in make test

[codecov]

Codecov Report

Patch coverage: 89.28% and project coverage change: -0.15 ⚠️

Comparison is base (76fe941) 66.72% compared to head (c036d67) 66.57%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1128      +/-   ##
==========================================
- Coverage   66.72%   66.57%   -0.15%     
==========================================
  Files         267      277      +10     
  Lines       29319    29999     +680     
==========================================
+ Hits        19562    19971     +409     
- Misses       8366     8601     +235     
- Partials     1391     1427      +36     

Flag	Coverage Δ
unittests	66.57% <89.28%> (-0.15%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...uler/controllers/migration/evictor/evictor_soft.go	89.28% <89.28%> (ø)

... and 42 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[eahydra]

Hi @leason00, thanks for your contribution.

Do we need to implement a new Evict Plugin? For issue #1102, my original idea was to add a new implementation to the evictor in PodMigrationController. Users can enable this new evictor implementation in MigrationControllerArgs.

In addition, we need to define the annotation added when the evict is executed, and define it as an API in apis/extension/descheduling.go , for example, scheduling.koordinator.sh/eviction=true means triggering eviction. In addition, you may have to consider adding some additional information, such as the time to trigger the eviction, the reason, who triggered it, and whether to set gracePeriodSeconds and forceDeletion, etc.

I think maybe a simple proposal can be given to clearly define this new change and API.

[hormes]

Do you want to reuse other logic in MigrationController?

[leason00]

It is better to reuse MigrationController. I'm going to redesign the MigrationController-based approach.

[leason00]

@eahydra Resubmitted. Review it again when you have time.

[hormes]

nice work!

[hormes]

/lgtm

[FillZpp]

/approve

[koordinator-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: FillZpp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• apis/OWNERS [FillZpp]
• pkg/descheduler/OWNERS [FillZpp]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[eahydra]

/lgtm