Introduce new time-window trigger

[tomkerkhove]

Proposal

Introduce new time-window trigger which allows you to scale based on a defined time window.

We could use inspiration from the kube-green project:

triggers:
- type: time-window
  weekdays: "Mo-Fri"
  start: "08:00"
  end: "20:00"
  timezone: Asia/Kolkata  # The acceptable values would be a value from the IANA Time Zone Database.

The cron scaler uses desiredReplicas but I believe it should just rely on min/max replica defined on the ScaledObject/ScaledJob.

This trigger should replace the cron scaler which is confusing in how it scales as end-users tend to think it behaves like jobs.

Use-Case

Scale workloads based on a time window.

Anything else?

No response

[yelghali]

thanks Tom, i think it's a useful feature. Maybe we should keep desiredReplicas or some other way to specify the target behavior

• during the time window, I (as an Admin) could need to scale down to min, during that window (say during weekends)
• or in a another scenario, I'd want to scale to maximum because i know i would have a predictible spike during that window (say for ecommerce)

In terms of user / customer scenarios, I understand this "time window" trigger / scaler should be used as the only scaler for the ScaledObject/Job (not in multiple triggers scenario, as of now)

[tomkerkhove]

thanks Tom, i think it's a useful feature. Maybe we should keep desiredReplicas or some other way to specify the target behavior

• during the time window, I (as an Admin) could need to scale down to min, during that window (say during weekends)
• or in a another scenario, I'd want to scale to maximum because i know i would have a predictible spike during that window (say for e-commerce)

How would you compare that to the already existing minReplicaCount & maxReplicaCount then? I think desiredReplicas would be redundant.

[yelghali]

thanks Tom, i think it's a useful feature. Maybe we should keep desiredReplicas or some other way to specify the target behavior

• during the time window, I (as an Admin) could need to scale down to min, during that window (say during weekends)
• or in a another scenario, I'd want to scale to maximum because i know i would have a predictible spike during that window (say for e-commerce)

How would you compare that to the already existing minReplicaCount & maxReplicaCount then? I think desiredReplicas would be redundant.

perhaps something like:

• desiredReplicaCount = max
• desiredReplicaCount = min

what I couldn't see, is how to tell the difference between when to scale In Or Out, during this window

[tomkerkhove]

Hm so during a given time window, you might want to scale in instead of out then? Why not invert the window then? To me it feels more natural to always scale to max during window and min outside of window.

[yelghali]

makes sens, as long as we define the rule (what you just did). I was seeing it as a time window where I could do both

[tomkerkhove]

We can do it, but I think it just makes it more confusing/redundant with the existing configuration already available

[zroubalik]

To me it feels more natural to always scale to max during window and min outside of window.

+1 to this approach. I'd extend cron scaler to do this

[tomkerkhove]

@zroubalik The proposal is actually to introduce a new scaler and deprecate the cron-scaler.
See issue description for context

[zroubalik]

okay, good to me as well.

[JorTurFer]

new scaler is a good idea to me, but I'd use cron expressions to start and end because it's more flexible

[tomkerkhove]

What scenario would you need that does not fit the above? Specify the exact second? Hour? Minute? or?

[JorTurFer]

During X time every hour for instance. A real user case (from my previous work):
Every hour a process generates KPIs scraping data from several places with heavy actions, using cron expressions you can specify that every hour from 0 to 15 with a single trigger instead of one trigger per hour being ready for the heavy load before it starts

[tomkerkhove]

Then I'd argue that we should not deprecate the cron scaler and just add it next to it, no?

[amirschw]

I'd argue that we should not deprecate the cron scaler and just add it next to it, no?

The cron trigger seems to be more flexible, both because of the use of cron format and the ability to set desiredReplicas independently of min/max replicas. I don't think that the proposed scaler can fully replace cron (at least not for our use cases), so +1 for keeping them side by side or extending cron as @zroubalik suggested.

[JorTurFer]

I don't have any strong opinion about both solutions. Maybe we could extend cron trigger or create another one. I'm not sure about the implementation because right now scalers don't have access to min/max values from SO/SJ and I'm not sure if they should know that info, but in any case it's more related with the implementation than with having or not other scaler

[tomkerkhove]

The cron trigger seems to be more flexible, both because of the use of cron format and the ability to set desiredReplicas independently of min/max replicas. I don't think that the proposed scaler can fully replace cron (at least not for our use cases), so +1 for keeping them side by side or extending cron as @zroubalik suggested.

@amirschw I'm curious about this remark because this creates a lot of confusion for the majority of people. Why would you want to specify desiredReplicas and how do you see it relate to min/max replicas which is redundant?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[tomkerkhove]

This is still confusing people :) I think we should think about this again @zroubalik @JorTurFer

[JorTurFer]

About introducing it or about the scaler itself?

[raorugan]

thanks Tom for adding this. I think this is a useful feature. It will be good to support NCRONTAB for the schedule and have the ability to set min/max replicas of target workload deployments.

[tomkerkhove]

@JorTurFer The new scaler because our cron scaler is confusing as people want to do CronJob-like things while all we do is scale based on defined time windows..

[JorTurFer]

I meant, we agreed with the new scaler, but nobody has tackled it yet.
@zroubalik agreed and I don't have an opinion about it :)

[tomkerkhove]

I don't think we have a agreement on how it relates to cron scaler and plan going forward though

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed due to inactivity.

[hajdukda]

Cron scaler reads as follows "not gonna work as a reoccuring schedule" ?

It's a REALLY REALLY common pattern that you have day/night or weekends or holidays that have lower traffic and it would make sense to scale down during this period. Many workloads are predictable.

Any plans to add this scaler ?

I would like to scale based on metrics between 9to5 and scale in to min (or even min-X) between 5to9 ? What options I currently have when using KEDA ?

[JorTurFer]

I would like to scale based on metrics between 9to5 and scale in to min (or even min-X) between 5to9 ? What options I currently have when using KEDA ?

I don't know what they used who added the comment meant, but using cron scaler you can achieve that behaviour. You could set a cron scaler which starts at 5 and ends at 9 with a desired value of X, and during that period you will have at least X.

Something that you cannot achieve is to use the cron scaler for scaling to X replicas if there is any other scaler requiring more replicas because the HPA controller does a MAX between all the metrics

[sbdtu5498]

@JorTurFer please assign after reviewing the conversation on issue #4677 if everything looks good.

[JorTurFer]

nice @sbdtu5498 ❤️
But remember to implement as a new scaler, cron scaler should be backward compatible

[v-shenoy]

@krishna-kariya PTAL.

[bharathguvvala]

This is a much needed feature. There are usecases where a feature like this would be useful eg. during planned spiky events, where the a deployment (or multiple) need to be scaled to maximum or a desired number.

Instead of dotting the scaledobjects with many cron scalers which is cumbersome for the individual managing the scaledobject and error prone IMO, this can be implemented as a different scaler coupled with #4459 . Further these planned events can have a widespread effect across multiple deployments (eg: an ecommerce company where a sale is scheduled and all the workloads need to scale to their maximum footprint) so expecting every dev to manage the time schedules for their workload is not desirable.

At our company we have implemented this on a fork of KEDA and has been in production since almost a year. If it seems relevant to most of the users we can work on the proposal.

[raorugan]

We are planning to introduce a new timer external scaler to check for NCRONTAB schedule and scale the pods from 0 to 1 and back to 0.

[tomkerkhove]

It is important to mention, however, that this new timer external scaler will have a dependency on Azure Storage and is for Azure Functions; no @raorugan? :)

[neoakris]

Hi I wanted to share a more concrete use case where this would be useful:

Basically:
desired count = (target each replica handling 100 requests per sec) + (additional replicas from cron)

^-- This would effectively make it so you could have a dynamically changing minimum value of replicas throughout the day. (not by adjusting the defined min, but because the reoccurring cron would enforce a min value on the dynamic desired count)

Also, I know you can't currently add metric counters like that (#2440 has a feature req for that, but even without 2440), you can do the equivalent of the above [desired = A + B] by defining both triggers and letting the built-in OR logic take place.
Basically

kind: ScaledObject
desiredReplicaCount:
- trigger 1: request per sec (from Prometheus)
- trigger 2: reoccuring cron

Here's a more concrete explanation of why reoccuring cron to set dynamicly changing min value could be useful:
If you have a app with spiky traffic and it's normal for you to need 200 replicas to serve traffic during peak periods of the day. and would be fine with only 3 replicas to serve traffic at night.
Let's say each replica can actually handle 150 RPS without any slowdown and 200 RPS without any errors.
If your autoscaling target is 100 RPS, then you kind of have an error buffer of 100 RPS per replica.
This means that more replicas = higher buffer --implies--> more replicas are better able to handle a traffic spike.

A business might have a regular traffic spike during certain business hours.
Currently, their only option is to set min replicas to 30 to give enough buffer to handle spikes, but they really only need that buffer during business hours. Then after hours they'd expect lower traffic spikes and could safely lower their baseline. Allowing a dynamic minimum value that adjusts each weekday / weekend could save money.

[JorTurFer]

@neoakris , #2440 has just been merged and it'll be released as experimental on next version (next week indeed xD)

[tomkerkhove]

I think we need to prioritize this one and kill CRON scaler, @zroubalik / @JorTurFer. It is confusing too many people as CRON is not a CRON