Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After upgrading Keda to v2.12.0 the v1beta1.external.metrics.k8s.io API component is not getting updated and its out of sync. #550

Open
vermaprateek695 opened this issue Oct 19, 2023 · 5 comments
Labels
bug Something isn't working

Comments

@vermaprateek695
Copy link

vermaprateek695 commented Oct 19, 2023

Expected Behavior

After upgrading Keda to v2.12.0 all the components should be in sync .

Actual Behavior

After upgrading Keda to v2.12.0 the v1beta1.external.metrics.k8s.io API component is not getting updated and it shows out of sync. When looking at the desired and the acutal state of the manifest file we could see the insecureSkipTLSVerify: false this value in desrired manifest is not getting picked up by argocd

image

PR where insecureSkipTLSVerify: false was added explicitly https://github.com/kedacore/charts/pull/520/files

Steps to Reproduce the Problem

  1. Upgraded the Keda version to v2.12.0
  2. synced the changes with Argocd
  3. v1beta1.external.metrics.k8s.io API component is not getting synced and shows outofsync with difference in the manifest .

Specifications

  • KEDA Version: * v2.12.0
  • Platform & Version: Please elaborate
  • Kubernetes Version: 1.28
  • Scaler(s): Please elaborate

image

can you please suggest any workaround which can fix this issue as in production env the Keda application shows outofsync due to this issue and this needs to be resolved at earliest.

@vermaprateek695 vermaprateek695 added the bug Something isn't working label Oct 19, 2023
@vermaprateek695 vermaprateek695 changed the title After upgrading Keda to v2.12.0 the v1beta1.external.metrics.k8s.io component is not getting updated and its out of sync. After upgrading Keda to v2.12.0 the v1beta1.external.metrics.k8s.io API component is not getting updated and its out of sync. Oct 19, 2023
@FutureMatt
Copy link

FutureMatt commented Oct 19, 2023

Yup, we have the same issue in our ArgoCD. I think the issue here is that it is trying to enforce the default value, so it never appears in the resource (do kubectl get apiservice v1beta1.external.metrics.k8s.io -o yaml and you'll see it isn't there), as a result ArgoCD see this as a difference in the resource manifests. I see a few options here:

  1. Wait it out until it eventually gets removed in a later Helm release
  2. Add a toggle to Helm to enable/disable this via a value
  3. Add that value to ArgoCD's ignore list (this is what we'll probably do)

@sidewinder12s
Copy link

Also ran into this, since the value is default and is not configurable in the helm chart should probably just remove it from the manifest. I didn't quite get what the change was fixing during upgrades, though ArgoCD might not have seen this issue since it doesn't use the normal Helm upgrade workflow.

If someone runs into this with ArgoCD you can use this ignoreDifferences config:

ignoreDifferences:
  - group: apiregistration.k8s.io
    kind: APIService
    jsonPointers:
      - /spec/insecureSkipTLSVerify

@zroubalik
Copy link
Member

@sidewinder12s could you please update our Troubleshooting section in docs with this info?

https://keda.sh/docs/2.12/troubleshooting/
https://github.com/kedacore/keda-docs/tree/main/content/troubleshooting

@sidewinder12s
Copy link

Actually turns out ArgoCD does need to apply the manifest as is in the chart. Once the upgrade is through you can apply the ignoreDifferences config however.

So I think users can either ignore it and wait for the workaround to be removed from the chart or use the ignoreDifferences config. I don't think this issue is very important to flag on general troubleshooting for Keda. Either way I can't commit to OSS repos without going through a legal review with my company.

@NasAmin
Copy link

NasAmin commented Apr 22, 2024

Has this been fixed in v2.13?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants