add ingressClass option to helm chart - back compatibility with ingress.class annotations

[tombokombo]

What this PR does / why we need it:

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation only

Which issue/s this PR fixes

Hi, according networking.k8s.io/v1 specification for ingressClassName

DESCRIPTION:
IngressClassName is the name of the IngressClass cluster resource. The
associated IngressClass defines which controller will implement the
resource. This replaces the deprecated kubernetes.io/ingress.class
annotation. For backwards compatibility, when that annotation is set, it
must be given precedence over this field.

and

https://kubernetes.github.io/ingress-nginx/#i-have-more-than-one-controller-in-my-cluster-and-already-use-the-annotation
I have more than one controller in my cluster and already use the annotation ?
No problem. This should still keep working, but we highly recommend you to test!

it gives impression that ingress.class annotation should still work "out-of-box". But is not, we need to specify --ingress-class argument for controller, which must be provided as extraArgs. I suggest to bring back .Values.controller.ingressClass as it was in 3.X and older charts. This could be done at least for chart version in 4.0.X for smoother chart upgrades and nondisruptive migration to IngressClass and respective IngressClassName field.
Thx

How Has This Been Tested?

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I've read the CONTRIBUTION guide
• I have added tests to cover my changes.
• All new and existing tests passed.

[k8s-ci-robot]

@tombokombo: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

Hi @tombokombo. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[longwuyuan]

@tombokombo , thank you for your contribution. It helps.

(1) Please create a issue first and move the description you have sown here to that issue.
(2) Link that issue to this PR
(3) Write tests and show your test config, data & results for other use cases after your sugested change.

The topic you have raised could be much more complicated in terms of the design, implementation, current state of the code and future roadmap. Potentially there could be discussions involved and comments from others. Hence tracking all that on a issue will help.

If you are a developer, please also look at the code and make any comments on the current implementation, with regard to the change you are sugesting. Thanks.

[rikatz]

@longwuyuan let me intervene on this one :D

We've been struggling a lot with this ingress class things, and how this thing reflected to the users.

Right now I understand that the proposed PR rollbacks something that we "deprecated" on past and "undeprecated" when we added the "ingress-class-by-name" so it is something missing on the helm charts.

The ingress class problem is much more complicated that it seems, and I've been really thinking on rollback all the deprecations we followed and say "ok, we are going to support again annotations OR ingressClass and wont say that annotations are deprecated anymore".

This probably is going to solve a lot of headaches we've had recently. This discussion was ongoing on sig-network in Slack, and my PERSONAL opinion is that although we need an ingressclass object, probably our timing wasn't good and it is what it is now, with a lot of broken deployments :) As far as I can see from GatewayAPI development, this wont be a problem

I wont be present on our next ingress-nginx meeting as I have a community event here, BUT I really want to bring this discussion back.

@strongjz @ElvinEfendi we should discuss this, I know the effort for ingressclass from kubernetes api and from our side was huge, but we need to make it better for our users (I'm saying from my side specifically!)

[rikatz]

/ok-to-test
/approve
/lgtm
About the PR, I'm not opposed to this.

[rikatz]

/lgtm cancel

[rikatz]

@tombokombo we need to update the helm docs now when a new option is added. Please see: https://github.com/kubernetes/ingress-nginx/blob/main/RELEASE.md#d-edit-the-valuesyaml-and-run-helm-docs

@longwuyuan as we are using now helm-docs it would be good if in Makefile we have a "make helm-docs" that downloads the release from https://github.com/norwoodj/helm-docs/releases/tag/v1.5.0 (or the latest available) and generates all for the users. Can you tackle on that with someone?

Thanks!

[rikatz]

/approve
Please fix the helm docs and we can merge it :)

[longwuyuan]

@tombokombo , based on @rikatz comments, let me know if you want help with running helm-docs on your clone, of your fork, to commit a updated /charts/ingress-nginx/README.md in your clone. That will potentially circumvent some confusion we have right now in the helm-docs part of the CI pipeline and thus get this PR ready for merging.

[tombokombo]

@longwuyuan now, it should be Ok.

[longwuyuan]

@rikatz , I think we need to use that bot option to squash commits here. Unless @tombokombo you want to squash commits in your fork

[tombokombo]

@longwuyuan ok, squashed.

[longwuyuan]

Thank you very much for squashing. Thank you very much for contribution.
@tombokombo , I am not sure if you are expected to bump chart version. That is part of the RELEASE process, afaik. @rikatz is it ok for this PR to bump chart version ?

Also, have been keen on improving the changelog so can you retitle your PR to something more co-herent like "add ingress.class annotation to helm chart" or something of that kind.

[tombokombo]

@longwuyuan title changed and regarding bumping chart vers, some ci expects it, some not, but as I remember correctly, I've checked some merged PR here and there was bump...

[longwuyuan]

True. The PRs created as part of the RELEASE.md process will bump the Chart version as documented in RELEASE.md.
AFAIK, if Chart version is bumped by you, then I have to check if that results in the release of a new Chart version or the bumped version just sits there until the "documented" RELEASE process occurs

[rikatz]

Let's release this :) I'm good with the bump

Just be aware the badge on the README.md seems to be pointing to the wrong chart version (or I may be wrong, which happens a lot when I'm reviewing PRs on cell phone)

@longwuyuan feel free to follow on this and lgtm

/approve
Thanks

[longwuyuan]

ok. @tombokombo please rebase if needed and run helm-docs once again, in your local clone. @rikatz comment is very important about the badge in README.md and AFAIK, that badge updates only and only after a certain process. I forgot how the badge changed (was after helmdocs or was it after release not sure).

Please feel free to ping me on slack and I can help do the needful to get this merged asap. This will help a lot of users.

[longwuyuan]

@tombokombo rebase is needed now

[tombokombo]

@longwuyuan rebased

[longwuyuan]

Thanks. Can you run helm-docs on your laptop clone as per documentation in RELEASE.md please. I am on slack also if you want. Thanks, ; Long

…

On Sun, 30 Jan, 2022, 6:28 PM Tomas Hulata, ***@***.***> wrote: @longwuyuan <https://github.com/longwuyuan> rebased — Reply to this email directly, view it on GitHub <#8136 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGZVWVOWPADHFBQM4WQKVLUYUYX3ANCNFSM5L2HM6DA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[tombokombo]

@longwuyuan docs fixed

[longwuyuan]

Thank you very much. We need to add. 16 and. 17 changelog and then do a helmchart release. Hopefully we will be able to do that soon. Regards. Thanks, ; Long

…

On Sun, 30 Jan, 2022, 8:11 PM Tomas Hulata, ***@***.***> wrote: @longwuyuan <https://github.com/longwuyuan> docs fixed — Reply to this email directly, view it on GitHub <#8136 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGZVWRKVFCNN6VHA6PY2W3UYVEYJANCNFSM5L2HM6DA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[bmv126]

@rikatz / @tombokombo
So now with this change is controller.ingressClassByName still needed in values.yaml. If yes, when do we use it and it's use case?

[tombokombo]

@rikatz / @tombokombo So now with this change is controller.ingressClassByName still needed in values.yaml. If yes, when do we use it and it's use case?

I will try to explain:

ingressClass option is way how to support Ingress object annotations.

IngressClassByName option is related to IngressClass object, there you are defining relation between ingressClassName field in Ingress object and ingress-controller. Controller should be specified in ingressClass spec.controller field, but some 3.X versions of helm chart has this field hard-coded and unfortunately this field is immutable ( https://github.com/kubernetes/ingress-nginx/blob/helm-chart-3.34.0/charts/ingress-nginx/templates/controller-ingressclass.yaml#L21 ). Some deployments are stucked with this ingressClass definition and they need ingressClassByName till they somehow solve this ( eg. delete and recreate ingressClass object ).

[rikatz]

/lgtm
Thanks

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rikatz, tombokombo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• charts/ingress-nginx/OWNERS [rikatz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment