External-DNS Chart Broken with 7.3.3 due to removal of protected CRD group annotation

[pinkfloydx33]

Name and Version

bitnami/external-dns

What architecture are you using?

None

What steps will reproduce the bug?

Attempt to upgrade or install the external-dns chart at version 7.3.3. In our case, an automated upgrade via Flux has started spamming alerts across all of our environments where minor/patch upgrades are automatically performed.

What is the expected behavior?

Helm chart upgrades/installs

What do you see instead?

Installation fails with:

Helm upgrade failed: cannot patch "dnsendpoints.externaldns.k8s.io" with kind CustomResourceDefinition: CustomResourceDefinition.apiextensions.k8s.io "dnsendpoints.externaldns.k8s.io" is invalid: metadata.annotations[api-approved.kubernetes.io]: Required value: protected groups must have approval annotation "api-approved.kubernetes.io", see kubernetes/enhancements#1111

Additional information

This is because the automated upgrade has removed the protected annotation on the CRDs:

• Commit @ Line12
• PR

This annotation is required on CRDs if the group is k8s.io, kubernetes.io, or ends with .k8s.io, .kubernetes.io which is applicable here because the CRD group is apiextensions.k8s.io.

This change needs to be reverted or else the chart is unusable.

[carrodher]

Thank you for bringing this issue to our attention. We appreciate your involvement! If you're interested in contributing a solution, we welcome you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance.

[pinkfloydx33]

@carrodher I wouldn't know where to begin as the original change was made by some form of automation, which tells me that it too would need to be fixed. I also unfortunately need approval from my company before contributing, even to tools we use, and that would likely take more time than others broken by this change would appreciate.

[andeke07]

I have added the annotation to the pull request on the original external-dns project, I believe this is where the Bitnami automation gets its information from so if this gets approved hopefully the next release will contain it again?

[andeke07]

This has been fixed now with kubernetes-sigs/external-dns@f46676f

I don't know the next steps though, I suppose a release needs to be done and then the bitnami chart updated to refer to the new release?

[rouke-broersma]

I think the release of such updates is automated, not sure how they are triggered but I expect that it will come within a reasonable time.

[carrodher]

Our automation looks for new releases at https://github.com/kubernetes-incubator/external-dns. When a new release is cut there, our automated test & release process is triggered. As part of that process, the upstream CRDs are compared and the ones in the Bitnami chart are updated to match the upstream, i.e 375ee3b

[herrbpl]

Still broken in 7.5.0

[andeke07]

external-dns hasn't issued a new release yet so there's nothing for the new chart to go off of

[raiomarco]

So... there's a fix? a workaround? if not, what version of the chart should i use? i tried the 7.5.2 but it didn't work :(

[cheinema]

@raiomarco Since the problem seems to have occurred starting with chart version 7.3.3, v7.3.2 should be the last stable version for now. We are still waiting for a new release in https://github.com/kubernetes-sigs/external-dns to include the fix.

[MaxAnderson95]

@cheinema Unless I'm missing something, the v0.14.2 release appears to have the fix. Is there anything else that needs to be done before merging the fix into the chart?

[rouke-broersma]

@cheinema Unless I'm missing something, the v0.14.2 release appears to have the fix. Is there anything else that needs to be done before merging the fix into the chart?

You're looking at the external dns helm chart managed by external dns. This is not the source of crds for the bitnami chart.

[MaxAnderson95]

@rouke-broersma I knew I was missing something! Thanks.