feat: deleted the pods that are not unmanaged by Cilium

[lvyanru8200]

Set operator to remove the label of a pod that existed before the node taint

1. Delete the specified label pod according to the parameter --pod-restart-selector, default value is k8s-app=kube-dns
2. --pod-restart-selector="" Remove all pods

Fixes: #21594
Signed-off-by: tigerK yanru.lv@daocloud.io

Please ensure your pull request adheres to the following guidelines:

• For first time contributors, read Submitting a pull request
• All code is covered by unit and/or runtime tests where feasible.
• All commits contain a well written commit description including a title,
  description and a Fixes: #XXX line if the commit addresses a particular
  GitHub issue.
• If your commit description contains a Fixes: <commit-id> tag, then
  please add the commit author[s] as reviewer[s] to this issue.
• All commits are signed off. See the section Developer’s Certificate of Origin
• Provide a title or release-note blurb suitable for the release notes.
• Thanks for contributing!

Fixes: #issue-number

Allow Cilium Operator to restart any unmanaged pods via --pod-restart-selector, rather than just kube-dns pods 

[lvyanru8200]

@joestringer @christarazi @aanm @nebril Sorry, please see this,My last pr time was too long there were a lot of other commissions between the two commissions

[christarazi]

/test

[owais]

Will this make it into 1.13.1? If so, do we know approximately when 1.13.1 will be released? Thanks!

[christarazi]

The feature was merged for 1.14 release and I don't see any backport labels for 1.13.

[owais]

Thanks @christarazi. I see 1.13.1 is being prepped already. We are running into similar issues with cilium adoption and this feature would help a lot. Any chance this can make it to 1.13.2? 😬

[williamclot]

I'm trying this feature with the 1.14.0-snapshot.1 release. I'm deploying Cilium using helm and have enabled this option using the extraArgs helm value (as I don't see a specific helm value for this option):

operator:
  extraArgs: ["--pod-restart-selector="]

This seems to be picked up properly as I see in the logs at the operator startup:

level=info msg=" --pod-restart-selector=''" subsys=cilium-operator-generic
level=info msg=" --unmanaged-pod-watcher-interval='15'" subsys=cilium-operator-generic

And yet unmanaged pods aren't being restarted from what I can see. When I run cilium status I still have only 40/43 pods managed by Cilium. Am I missing something to make this work?

[christarazi]

@williamclot If you believe it's a bug, please file an issue and provide reproduction steps.