Support evict annotation for namespaces

[damemi]

The operator currently auto-excludes all namespaces with openshift-* or kube-* prefixes from eviction. This makes sense to prevent users from breaking their cluster with the Descheduler, and those are reserved prefixes so users should not be able to create their own namespaces that match the pattern.

However, it may be useful for administrators and support to be able to include certain system namespaces for rebalancing (for example, during and after upgrades). Perhaps we could add a check for the same descheduler.alpha.kubernetes.io/evict annotation on namespaces before assuming they should be excluded. Pods within that namespace would still be subject to the same eviction rules

cc @ingvagabund wdyt?

[ingvagabund]

We might also introduce a new profile which will do this right before/after the upgrade if this is the only use case. Make it part of the upgrade itself (pre/post-upgrade steps).

[ravitri]

@damemi , @ingvagabund - In addition to the upgrades, the other scenarios I can think of which could possibly disrupt existing core workloads is node/machine replacements/addition/removal. For now, the worker and infra nodes are supported as part of this but in future will be extended to control plane too (subject to change). Another scenario can be a MachineConfig change to cause rolling reboot of nodes in the respective MachineConfigPool (triggered change which is not an upgrade).

Considering above scenarios as well, I think we might need to stretch openshift* namespace inclusion criteria further. WDYT?

[wking]

Descheduler is very polite by using the eviction API. We have efforts like openshift/origin#26160 underway to improve our PDB coverage. If folks using the eviction API can cause excessive disruption in the OpenShift core, that sounds like it's really a missing/miscongured PDB situation to me. I expect we have some bugs like that today. Hopefully openshift/origin#26160 turns them up, and we get them fixed. Once we get them fixed, can we pivot to having the descheduler cover the kube-system and openshift-* namespaces by default? Because "don't ask about evicting us, we don't handle that well" doesn't seem like a good long-term plan.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.