[helm] ingress-secret isn't idempotent

[TheRealNoob]

Is your feature request related to a problem? Please describe.

The core issue is that it doesn't seem to be possible right now to render an idempotent ingress-secret. This matters because CD tools such as Argocd or Flux mandate this -- they can't compare desired state against currently deployed state if desired state changes every time its rendered. Ultimately this is a limitation of the helm implementation of htpasswd which uses the bcrypt algorithm that adds salt to the password which is why the output is always unique. Both a pro and a con. The htpasswd binary uses MD5 by default which doesn't use salt, but that's not the algorithm the helm implementation is using.

Describe the solution you'd like

This solution works but only for some ingress controllers. ingress-nginx supports an alternative format for basic-auth using the annotation nginx.ingress.kubernetes.io/auth-secret-type: auth-map (which is actually already defined) which means that each username will be stored as a key, and password as the b64enc value. This must be special handling by the ingress-nginx folks as I see no mention of it in nginx-ingress and native nginx doesn't support this which means gateway won't. An example of how this could be done: TheRealNoob@525f44b

I wasn't able to find a solution that would work for gateway given that native nginx only supports encrypted passwords that came out of the cryptography library. Not unless you got into solutions involving init containers passing the b64enc values (as above) through htpasswd. That way the k8s object is technically idempotent.

Describe alternatives you've considered

Additional context

Please let me know your thoughts on this approach -- as stated it won't work for gateway but it seems to be a harmless change allowing ingress to at least work.