Updating scaledObject annotations does not propagate the to HPA

[chubchubsancho]

Report

We need some annotations on our HPA for internal process. And sometimes we need to update them.
Acutally scaledObject annotation are propagated on creation but they are not updated if scaledObject change target only annotiations.

If other change append at same times no there is no problem

Expected Behavior

Even if we update only annotations on scaledObject hpa need to be reconcile.

Actual Behavior

No change append on hpa annotations

Steps to Reproduce the Problem

1. Create a scaled object with some annotations

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  annotations:
    test: "this-is-a-test"
  labels:
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/version: 1.0.0
  name: test
spec:
  advanced:
    horizontalPodAutoscalerConfig:
      behavior:
        scaleDown:
          policies:
          - periodSeconds: 60
            type: Percent
            value: 10
          - periodSeconds: 60
            type: Pods
            value: 5
          selectPolicy: Min
          stabilizationWindowSeconds: 1800
        scaleUp:
          policies:
          - periodSeconds: 5
            type: Percent
            value: 100
          - periodSeconds: 5
            type: Pods
            value: 4
          selectPolicy: Max
          stabilizationWindowSeconds: 0
    scalingModifiers: {}
  cooldownPeriod: 300
  maxReplicaCount: 10
  minReplicaCount: 2
  pollingInterval: 30
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: test
  triggers:
  - metadata:
      value: "20"
    metricType: AverageValue
    type: cpu

3. HPA is created with those annotations
4. Update scaledObject annotations and deploy it

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  annotations:
    test: "this-is-another-test"
  labels:
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/version: 1.0.0
  name: test
spec:
  advanced:
    horizontalPodAutoscalerConfig:
      behavior:
        scaleDown:
          policies:
          - periodSeconds: 60
            type: Percent
            value: 10
          - periodSeconds: 60
            type: Pods
            value: 5
          selectPolicy: Min
          stabilizationWindowSeconds: 1800
        scaleUp:
          policies:
          - periodSeconds: 5
            type: Percent
            value: 100
          - periodSeconds: 5
            type: Pods
            value: 4
          selectPolicy: Max
          stabilizationWindowSeconds: 0
    scalingModifiers: {}
  cooldownPeriod: 300
  maxReplicaCount: 10
  minReplicaCount: 2
  pollingInterval: 30
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: test
  triggers:
  - metadata:
      value: "20"
    metricType: AverageValue
    type: cpu

6. HPA is not updated

Logs from KEDA operator

example

KEDA Version

2.13.0

Kubernetes Version

1.26

Platform

Other

Scaler Details

cpu,prometheus

Anything else?

No response

[JorTurFer]

Hey!
Thanks for reporting! I can replicate the issue.
Are you willing to contribute with the fix?

[JorTurFer]

I think that the issue is just that KEDA doesn't trigger a reconciliation if (generic) annotations or labels change:

keda/controllers/keda/scaledobject_controller.go

Lines 129 to 136 in cd57dd9

	For(&kedav1alpha1.ScaledObject{}, builder.WithPredicates(
	predicate.Or(
	kedacontrollerutil.PausedPredicate{},
	kedacontrollerutil.PausedReplicasPredicate{},
	kedacontrollerutil.ScaleObjectReadyConditionPredicate{},
	predicate.GenerationChangedPredicate{},
	),
	)).

Do you know if there is any reason for ignoring those changes @zroubalik ?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[zroubalik]

I think that the issue is just that KEDA doesn't trigger a reconciliation if (generic) annotations or labels change:

keda/controllers/keda/scaledobject_controller.go

Lines 129 to 136 in cd57dd9

	For(&kedav1alpha1.ScaledObject{}, builder.WithPredicates(
	predicate.Or(
	kedacontrollerutil.PausedPredicate{},
	kedacontrollerutil.PausedReplicasPredicate{},
	kedacontrollerutil.ScaleObjectReadyConditionPredicate{},
	predicate.GenerationChangedPredicate{},
	),
	)).

Do you know if there is any reason for ignoring those changes @zroubalik ?

Yeah, this has been done to reduce the number of reconciliations.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed due to inactivity.