Response to priority change of PropagationPolicy

[RainbowMango]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR provides a temporary solution for corner case:

After the priority(.spec.priority) of PropagationPolicy changed from high priority (e.g. 5) to 
low priority(e.g. 3), we should try to check if there is a PropagationPolicy(e.g. with priority 4) 
that could preempt the targeted resources.

Which issue(s) this PR fixes:
Fixes #
Part of #3787

Special notes for your reviewer:
See #3684 (comment) for the background discussion.

In addition, this PR focuses on dealing with PropagationPolicy, if it is acceptable, the ClusterPropagationPolicy should do a similar thing.

Test steps:

1. Apply two policies (foo with priority 5 and bar with priority 4) as follows

apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: foo
spec:
  priority: 5
  preemption: Always
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: nginx
  placement:
    clusterAffinity:
      clusterNames:
        - member1
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: bar
spec:
  priority: 4
  preemption: Always
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: nginx
  placement:
    clusterAffinity:
      clusterNames:
        - member2

2. Apply a Deployment as this sample.

kubectl apply -f samples/nginx/deployment.yaml

3. The Deployment should be propagated by foo as it has higher priority

apiVersion: apps/v1
kind: Deployment
metadata:
  generation: 1
  labels:
    app: nginx
    propagationpolicy.karmada.io/name: foo
    propagationpolicy.karmada.io/namespace: default
  name: nginx

4. Update the priority of foo from 5 to 3, then the Deployment should be preempted by bar.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
    propagationpolicy.karmada.io/name: bar
    propagationpolicy.karmada.io/namespace: default

meanwhile, I can see the relevant logs:

I0821 13:28:39.213404       1 preemption.go:263] Enqueuing PropagationPolicy(default/bar) in case of PropagationPolicy(default/foo) priority changes
...
I0821 13:28:39.226380       1 preemption.go:127] Propagation policy(default/bar) has preempted another propagation policy(default/foo).

Does this PR introduce a user-facing change?:

NONE(Part of feature #3787)

[codecov-commenter]

Codecov Report

Merging #3965 (90898e2) into master (5f16640) will decrease coverage by 0.16%.
Report is 4 commits behind head on master.
The diff coverage is 0.00%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@            Coverage Diff             @@
##           master    #3965      +/-   ##
==========================================
- Coverage   54.66%   54.50%   -0.16%     
==========================================
  Files         232      232              
  Lines       22521    22585      +64     
==========================================
- Hits        12311    12310       -1     
- Misses       9537     9602      +65     
  Partials      673      673              

Flag	Coverage Δ
unittests	54.50% <0.00%> (-0.16%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
pkg/detector/detector.go	0.00% <0.00%> (ø)
pkg/detector/preemption.go	0.00% <0.00%> (ø)

... and 1 file with indirect coverage changes

[RainbowMango]

@wuyingjun-lucky Do you have any comments? I see you are looking at it :)

[whitewindmills]

it's a good idea.

[wuyingjun-lucky]

LGTM and I want have a try on “multiple preemption”

[RainbowMango]

LGTM and I want have a try on “multiple preemption”

No, you don't, haha, that's a chaotic state(but will stabilize eventually.)

[whitewindmills]

/lgtm

[karmada-bot]

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment