Controller doesn't process request if stuck elsewhere (OIDC annotation missing secret)

[theintz]

Describe the bug
When an ingress change request can't be fulfilled by the controller, it logs the error and then stops processing all other requests, even those that could be fulfilled.

Steps to reproduce
Create two deployments with ingresses, one that will deploy without errors, another one that doesn't (for example by requiring external resources such as a secret that aren't available). If the faulty one is requested first, the controller will fail to deploy it and log errors repeatedly. When the second request is issued, it will not be processed, even though it is not connected to the faulty request and could be fulfilled.

More concretely, we are using the alb.ingress.kubernetes.io/auth-idp-oidc annotation, to set up the LB to do auth over OIDC. It requires passing a parameter called secretName containing the name of the k8s secret that holds the clientSecret as required by the Oauth2 standard. If this secret does not exist, the controller exhibits the mentioned behavior.

Expected outcome
Valid ingress change requests are processed, even if the controller is stuck on a faulty request elsewhere.

Environment

• AWS Load Balancer controller version
  1.3.2
• Kubernetes version
  1.20
• Using EKS (yes/no), if so version?
  yes, most current

[kishorj]

@theintz, in case of errors, controller will not reconcile the ingress or the group further. If you use ingress group feature, fatal errors encountered during processing of one ingress affects the entire group. Other ingresses not part of the group should still continue to work as expected.

Controller tolerates certain errors, for example if backend service doesn't exist, it configures a 404 response but doesn't block the reconciliation.

[nabadger]

Is there a more general solution/roadmap in this area to allow valid ingresses within the same group to reconcile if 1 or more are in an error state?

Seems somewhat similar to #2042

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[theintz]

/remove-lifecycle stale

[theintz]

@kishorj Thanks for the explanation and sorry for the late reply. I don't understand the motivation behind this behavior. Even if a number of ingresses share the same group, why would the reconciliation be blocked if one of them fails? Shouldn't the controller try to reconcile as many ingresses as possible?

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.