graceful-eviction does not work.

[chaunceyjiang]

What happened:
graceful-eviction does not work.

depoly.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 4
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: docker.m.daocloud.io/nginx
        name: nginx
        resources:
          limits:
            cpu: 250m
            memory: 512Mi
          requests:
            cpu: 250m
            memory: 512Mi

pp.yaml

apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: nginx-propagation
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: nginx
  placement:
    clusterAffinities:
      - affinityName: default
        clusterNames:
          - member1
          - member2
    replicaScheduling:
      replicaSchedulingType: Divided
      replicaDivisionPreference: Weighted
      weightPreference:
        dynamicWeight: AvailableReplicas

Add taints.

./_output/bin/darwin/arm64/karmadactl taint clusters member1 key1=value1:NoExecute 

RB.yaml

spec:
  clusters:
  - name: member2
    replicas: 4
  gracefulEvictionTasks:
  - fromCluster: member1
    producer: TaintManager
    reason: TaintUntolerated
    replicas: 2
  placement:
    clusterAffinities:
    - affinityName: default
      clusterNames:
      - member1
      - member2
    clusterTolerations:
    - effect: NoExecute
      key: cluster.karmada.io/not-ready
      operator: Exists
      tolerationSeconds: 30
    - effect: NoExecute
      key: cluster.karmada.io/unreachable
      operator: Exists
      tolerationSeconds: 30
    replicaScheduling:
      replicaDivisionPreference: Weighted
      replicaSchedulingType: Divided
      weightPreference:
        dynamicWeight: AvailableReplicas
  replicaRequirements:
    resourceRequest:
      cpu: 250m
      memory: 512Mi
  replicas: 4
  resource:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx
    namespace: default
    resourceVersion: "6942"
    uid: 78aff1f4-ddf4-40f9-9880-5d5f8ea23f33
  schedulerName: default-scheduler
status:
  aggregatedStatus:
  - applied: true
    clusterName: member1
    health: Healthy
    status:
      availableReplicas: 2
      readyReplicas: 2
      replicas: 2
      updatedReplicas: 2
  - applied: true
    clusterName: member2
    health: Healthy
    status:
      availableReplicas: 4
      readyReplicas: 4
      replicas: 4
      updatedReplicas: 4
  conditions:
  - lastTransitionTime: "2023-04-26T03:20:45Z"
    message: Binding has been scheduled
    reason: BindingScheduled
    status: "True"
    type: Scheduled
  - lastTransitionTime: "2023-04-26T03:20:46Z"
    message: All works have been successfully applied
    reason: FullyAppliedSuccess
    status: "True"
    type: FullyApplied
  schedulerObservedGeneration: 3
  schedulerObservingAffinityName: default

  gracefulEvictionTasks:
  - fromCluster: member1
    producer: TaintManager
    reason: TaintUntolerated
    replicas: 2

The gracefulEvictionTasks queue is never empty.

Restart karmada-controller-manager doesn't work either.

What you expected to happen:

graceful-eviction can work.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• Karmada version: release-1.5
• kubectl-karmada or karmadactl version (the result of kubectl-karmada version or karmadactl version):
• Others:

[RainbowMango]

Please @XiShanYongYe-Chang @jwcesign take a look if it is similar to #3456.

[jwcesign]

Hi, can you check if there is a log for this function?

karmada/pkg/controllers/gracefuleviction/rb_graceful_eviction_controller.go

Line 39 in 0f84dd7

klog.V(4).Infof("Reconciling ResourceBinding %s.", req.NamespacedName.String())

If there is not, #3455 will solve your problem.

[chaunceyjiang]

The root cause of this problem is that rb.status.SchedulerObservedGeneration does not equal rb.Generation.

https://github.com/karmada-io/karmada/blob/master/pkg/controllers/gracefuleviction/rb_graceful_eviction_controller.go#L95

[jwcesign]

...n does not equal rb.Generation.

Yes, @Poor12 is taking care of it.