Scale to zero with cron triggers #3609

gaganyaan2 · 2022-08-29T17:20:48Z

Proposal

Currently, We cannot achieve cron based scale down to zero with Keda.
If I apply below CRD it set nginx-deployment replicas=0 and It does not wait for triggers.metadata.start.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: nginx-deployment-keda
spec:
  minReplicaCount: 0
  maxReplicaCount: 1
  scaleTargetRef:
    name: nginx-deployment
  triggers:
  - type: cron
    metadata:
      timezone: Asia/Kolkata
      start: 0 0 * * 6 ## At 00:00 on Saturday
      end: 0 0 * * 7 ## At 00:00 on Sunday
      desiredReplicas: "0"

Keda Helm chart : Keda-2.8.1
Kubernetes: v1.20

Here are some previous discussions regarding the same issue #1759 #2153

Can we make the above manifest to set replicas=0 at the start of 00:00 on Saturday and set replicas=1 at the end of 00:00 on Sunday? Currently, it set nginx-deployment replicas=0 immediately after applying.

OR Below is a simple CRD example that would enable scaling down to zero and easier for a user to read and understand.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: nginx-deployment-keda
spec:
  scaleTargetRef:
    name: nginx-deployment
  triggers:
  - type: cron
    metadata:
      timezone: Asia/Kolkata
      start: 0 0 * * 6 ## At 00:00 on Saturday
        minReplicaCount: 0
        maxReplicaCount: 1
        desiredReplicas: 0
      end: 0 0 * * 7 ## At 00:00 on Sunday
        minReplicaCount: 0
        maxReplicaCount: 1
        desiredReplicas: 0

Or Rather than using HPA if we could directly update replicas by using the below example CRD that would be great.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: nginx-deployment-keda
spec:
  scaleTargetRef:
    name: nginx-deployment
  triggers:
  - type: cron
    metadata:
      timezone: Asia/Kolkata
      start: 0 0 * * 6    ## At 00:00 on Saturday
        replicas: 0
      end: 0 0 * * 7    ## At 00:00 on Sunday
        replicas: 1

Use-Case

Scale down deployment to zero during the weekend to save cost
Scale down lower environment deployment during the night time to save cost

Anything else?

No response

The text was updated successfully, but these errors were encountered:

JorTurFer · 2022-09-26T19:32:48Z

Hey,
I'd say that behaviour is doable, just using multiple cron triggers in the same SO. I mean, if you need to specify different values for different cases, you could just use multiple triggers.
The only point here, is that the HPA Controller will take the MAX value between all the metrics, so if you want to scale to zero, all triggers should be outside the enabled period

tomkerkhove · 2022-09-27T06:43:02Z

I think the cron scaler is just confusing as it only scales to the desired replicas and ignores majority of replica count on the metadata directly.

This proposal, to be honest, would make that even worse/more confusing (no offense):

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: nginx-deployment-keda
spec:
  scaleTargetRef:
    name: nginx-deployment
  triggers:
  - type: cron
    metadata:
      timezone: Asia/Kolkata
      start: 0 0 * * 6 ## At 00:00 on Saturday
        minReplicaCount: 0
        maxReplicaCount: 1
        desiredReplicas: 0
      end: 0 0 * * 7 ## At 00:00 on Sunday
        minReplicaCount: 0
        maxReplicaCount: 1
        desiredReplicas: 0

So in summary - You want to define a window and scale to x; outside of that window you want to scale to 0. Is that correct?

gaganyaan2 · 2022-09-27T13:00:04Z

@tomkerkhove yes correct.

scale_up = define a window and scale up to x
scale_down = define a window and scale down to 0

tomkerkhove · 2022-09-27T13:21:32Z

Then all you need is this AFAIK:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: nginx-deployment-keda
spec:
  minReplicaCount: 0
  scaleTargetRef:
    name: nginx-deployment
  triggers:
  - type: cron
    metadata:
      timezone: Asia/Kolkata
      start: 0 0 * * 6 ## At 00:00 on Saturday
      end: 0 0 * * 7 ## At 00:00 on Sunday
      desiredReplicas: "x"

tomkerkhove · 2022-09-27T13:21:51Z

(nit: scale_up -> scale_out; scale_down -> scale_in)

stale · 2022-11-26T14:56:17Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale · 2022-12-03T20:47:08Z

This issue has been automatically closed due to inactivity.

vipulp86 · 2022-12-05T09:45:43Z

@tomkerkhove that is the problem/bug with minReplicaCount used with cron scaled object, when you set minReplicaCount to 0(which is by default 0 too) and desiredReplicaCount to 0(as we want to auto scaled in to 0 during said cron), and as we apply scaledobject definition at first step itself its bringing down pods to 0. Which should not be the case as per definition minReplicaCount.

gaganyaan2 · 2023-04-25T07:25:45Z

@tomkerkhove I tried with keda version 2.8.2. still, as soon as I apply the below configuration ScaledObject it brings down the nginx deployment replicas to zero. ideally, it should only bring down during at start time.

Also at start time, it sets replicas=1 which is weird.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: nginx-keda
spec:
  minReplicaCount: 0
  scaleTargetRef:
    name: nginx
  pollingInterval: 10
  cooldownPeriod: 30
  minReplicaCount: 0
  triggers:
  - type: cron
    metadata:
      timezone: Asia/Kolkata
      start: 30 * * * *
      end: 35 * * * *
      desiredReplicas: "0"

as soon as I apply above scaleobject I get the below logs(keda-operator) it says scaling down the "New Replicas Count": 0

2023-04-25T06:11:48Z    INFO    Reconciling ScaledObject        {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "3340436d-a5cb-4cc4-b321-4010c23eeaa0"}
2023-04-25T06:11:48Z    INFO    Adding Finalizer for the ScaledObject   {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "3340436d-a5cb-4cc4-b321-4010c23eeaa0"}
2023-04-25T06:11:48Z    INFO    Detected resource targeted for scaling  {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "3340436d-a5cb-4cc4-b321-4010c23eeaa0", "resource": "apps/v1.Deployment", "name": "nginx"}
2023-04-25T06:11:48Z    INFO    Creating a new HPA      {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "3340436d-a5cb-4cc4-b321-4010c23eeaa0", "HPA.Namespace": "keda", "HPA.Name": "keda-hpa-nginx-keda"}
2023-04-25T06:11:48Z    INFO    Initializing Scaling logic according to ScaledObject Specification      {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "3340436d-a5cb-4cc4-b321-4010c23eeaa0"}
2023-04-25T06:11:48Z    INFO    Reconciling ScaledObject        {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "107beb1b-e1a3-409e-b84e-cfcafea95f65"}
2023-04-25T06:11:48Z    INFO    scaleexecutor   Successfully set ScaleTarget replicas count to ScaledObject minReplicaCount     {"scaledobject.Name": "nginx-keda", "scaledObject.Namespace": "keda", "scaleTarget.Name": "nginx", "Original Replicas Count": 1, "New Replicas Count": 0}
2023-04-25T06:11:48Z    INFO    Reconciling ScaledObject        {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "4504159c-de24-47db-bd6a-139d0c70c802"}
2023-04-25T06:12:03Z    INFO    Reconciling ScaledObject        {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "142f3092-b70f-4e1e-9571-0078d8cc33e3"}

HPA describe

$ k describe hpa keda-hpa-nginx-keda
Name:                                                           keda-hpa-nginx-keda
Namespace:                                                      keda
Labels:                                                         app.kubernetes.io/managed-by=keda-operator
                                                                app.kubernetes.io/name=keda-hpa-nginx-keda
                                                                app.kubernetes.io/part-of=nginx-keda
                                                                app.kubernetes.io/version=2.8.2
                                                                scaledobject.keda.sh/name=nginx-keda
Annotations:                                                    <none>
CreationTimestamp:                                              Tue, 25 Apr 2023 12:24:11 +0530
Reference:                                                      Deployment/nginx
Metrics:                                                        ( current / target )
  "s0-cron-Asia-Kolkata-42xxxx-47xxxx" (target average value):  <unknown> / 1
Min replicas:                                                   1
Max replicas:                                                   100
Deployment pods:                                                0 current / 0 desired
Conditions:
  Type           Status  Reason             Message
  ----           ------  ------             -------
  AbleToScale    True    SucceededGetScale  the HPA controller was able to get the target's current scale
  ScalingActive  False   ScalingDisabled    scaling is disabled since the replica count of the target is zero
Events:          <none>

This is after cron start time

2023-04-25T06:11:48Z    INFO    Reconciling ScaledObject        {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "4504159c-de24-47db-bd6a-139d0c70c802"}
2023-04-25T06:12:03Z    INFO    Reconciling ScaledObject        {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "142f3092-b70f-4e1e-9571-0078d8cc33e3"}
2023-04-25T06:15:08Z    INFO    scaleexecutor   Successfully updated ScaleTarget        {"scaledobject.Name": "nginx-keda", "scaledObject.Namespace": "keda", "scaleTarget.Name": "nginx", "Original Replicas Count": 0, "New Replicas Count": 1}
2023-04-25T06:15:18Z    INFO    Reconciling ScaledObject        {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "aec18199-20fb-4f7d-be9b-19658fbdc477"}
2023-04-25T06:20:03Z    INFO    Reconciling ScaledObject        {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "529c180f-587f-4ae4-8e22-0ce4f9527e34"}
2023-04-25T06:20:28Z    INFO    scaleexecutor   Successfully set ScaleTarget replicas count to ScaledObject minReplicaCount     {"scaledobject.Name": "nginx-keda", "scaledObject.Namespace": "keda", "scaleTarget.Name": "nginx", "Original Replicas Count": 1, "New Replicas Count": 0}
2023-04-25T06:20:34Z    INFO    Reconciling ScaledObject        {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "scaledObject": {"name":"nginx-keda","namespace":"keda"}, "namespace": "keda", "name": "nginx-keda", "reconcileID": "9d09c03d-3a5c-4eb6-89f7-800541a23b61"}

This is a diagram of what is actually happening vs what is expected.

Expected
Actual happening

Please help.

gaganyaan2 · 2023-04-25T10:53:18Z

Workaround - Make desired state between start and end timing #4474 (comment)

gaganyaan2 added feature-request All issues for new features that have not been committed to needs-discussion labels Aug 29, 2022

tomkerkhove added this to Roadmap - KEDA Core Aug 29, 2022

tomkerkhove moved this to Proposed in Roadmap - KEDA Core Aug 29, 2022

stale bot added the stale All issues that are marked as stale due to inactivity label Nov 26, 2022

stale bot closed this as completed Dec 3, 2022

Repository owner moved this from Proposed to Ready To Ship in Roadmap - KEDA Core Dec 3, 2022

JorTurFer moved this from Ready To Ship to Done in Roadmap - KEDA Core Mar 13, 2023

gaganyaan2 mentioned this issue Apr 25, 2023

bug - Scale to zero with cron triggers not working #4474

Closed

This was referenced Mar 8, 2024

cron scaler: clarify desiredReplicas 0 kedacore/keda-docs#1332

Merged

Cron Scaler: desiredReplicas is 0 but it sets to 1 anyway #5586

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scale to zero with cron triggers #3609

Scale to zero with cron triggers #3609

gaganyaan2 commented Aug 29, 2022 •

edited

Loading

JorTurFer commented Sep 26, 2022

tomkerkhove commented Sep 27, 2022

gaganyaan2 commented Sep 27, 2022

tomkerkhove commented Sep 27, 2022

tomkerkhove commented Sep 27, 2022

stale bot commented Nov 26, 2022

stale bot commented Dec 3, 2022

vipulp86 commented Dec 5, 2022

gaganyaan2 commented Apr 25, 2023

gaganyaan2 commented Apr 25, 2023

Scale to zero with cron triggers #3609

Scale to zero with cron triggers #3609

Comments

gaganyaan2 commented Aug 29, 2022 • edited Loading

Proposal

Use-Case

Anything else?

JorTurFer commented Sep 26, 2022

tomkerkhove commented Sep 27, 2022

gaganyaan2 commented Sep 27, 2022

tomkerkhove commented Sep 27, 2022

tomkerkhove commented Sep 27, 2022

stale bot commented Nov 26, 2022

stale bot commented Dec 3, 2022

vipulp86 commented Dec 5, 2022

gaganyaan2 commented Apr 25, 2023

gaganyaan2 commented Apr 25, 2023

gaganyaan2 commented Aug 29, 2022 •

edited

Loading