[ingress-nginx] migrate to the AdvancedDaemonSet controller

[yalosev]

Description

Use OpenKruise AdvancedDaemonSet controller instead of built-in.

Why do we need it, and what problem does it solve?

Kubernetes DaemonSet controller has some problems in the heavy-loaded environments, which are hard to debug. Kruise controller has more rich workflow and expected to have smooth ingress-controller rollout process

What is the expected result?

Checklist

• The code is covered by unit tests.
• e2e tests passed.
• Documentation updated according to the changes.
• Changes were tested in the Kubernetes cluster manually.

Changelog entries

section: ingress-nginx
type: feature
summary: Use `AdvancedDaemonSet` controller for smooth ingress-controller rollout.
impact: Ingress-controller Pods will restart.
impact_level: default

[deckhouse-BOaTswain]

🔴 e2e: Yandex.Cloud for deckhouse:ingress-ds-advanced failed in 57m44s.

E2e for yandex-cloud WithoutNAT;containerd;1.23 was failed. Use:
ssh -i ~/.ssh/e2e-id-rsa cloud-user@178.154.207.18 - connect for debugging;

/e2e/abort yandex-cloud;WithoutNAT;containerd;1.23 4564841121 4564841121-1-con-1-23 /sys/deckhouse-oss/install:pr4124 cloud-user@178.154.207.18 - for abort failed cluster

Workflow details (1 job failed)

Yandex.Cloud-WithoutNAT-Containerd-1.23 - Connection string: ssh cloud-user@178.154.207.18

🔴 e2e: Yandex.Cloud, Containerd, Kubernetes 1.23 failed in 26m32s.

[deckhouse-BOaTswain]

🟢 e2e: AWS for deckhouse:ingress-ds-advanced succeeded in 42m22s.

Workflow details

AWS-WithoutNAT-Containerd-1.23 - Connection string: ssh centos@18.198.213.136

🟢 e2e: AWS, Containerd, Kubernetes 1.23 succeeded in 41m45s.

[yalosev]

/e2e/abort yandex-cloud;WithoutNAT;containerd;1.23 4564841121 4564841121-1-con-1-23 /sys/deckhouse-oss/install:pr4124 cloud-user@178.154.207.18

[deckhouse-BOaTswain]

🟢 e2e destroy failed: Yandex.Cloud for refs/heads/main succeeded in 4m47s.

Workflow details

🟢 e2e destroy failed: Yandex.Cloud, Containerd, Kubernetes 1.23 succeeded in 4m29s.

[deckhouse-BOaTswain]

🟢 e2e: AWS for deckhouse:ingress-ds-advanced succeeded in 27m3s.

Workflow details

AWS-WithoutNAT-Containerd-1.23 - Connection string: ssh centos@3.76.183.197

🟢 e2e: AWS, Containerd, Kubernetes 1.23 succeeded in 26m22s.

[nabokihms]

I think I understand what is happening, yet I still left some comments to clarify details.

[deckhouse-BOaTswain]

🟢 e2e: AWS for deckhouse:ingress-ds-advanced succeeded in 29m28s.

Workflow details

AWS-WithoutNAT-Containerd-1.23 - Connection string: ssh centos@3.73.249.249

🟢 e2e: AWS, Containerd, Kubernetes 1.23 succeeded in 27m0s.