Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CronJobs should support timezones #47202

Closed
iterion opened this issue Jun 8, 2017 · 141 comments
Closed

CronJobs should support timezones #47202

iterion opened this issue Jun 8, 2017 · 141 comments
Assignees
Labels
area/batch area/workload-api/cronjob kind/feature Categorizes issue or PR as related to a new feature. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. sig/apps Categorizes an issue or PR as relevant to SIG Apps.

Comments

@iterion
Copy link
Contributor

iterion commented Jun 8, 2017

I couldn't find any discussion of this after searching for "cron timezone", "cronjob timezone", or "scheduledjob timezone".

The CronJob spec makes reference to https://en.wikipedia.org/wiki/Cron. That page suggests that cron would respect the timezone for a given user. The controller manager runs in a single time zone under a single user so I can't use different time zones for each job. I have jobs that run based on the schedule of external entities that observe daylight savings time. So, if I define that CronJob in UTC I will be forced to update that job from time to time (generally not something one remembers to do after just losing an hour of sleep).

I see two options for how this support might work in kubernetes:

  1. Add some new field to the CronJobSpec, like timezone: "Americas/Chicago".
  2. Use an extended cron syntax that includes timezone, e.g. 30 6 * * 1 Europe/Stockholm
@k8s-github-robot
Copy link

@iterion There are no sig labels on this issue. Please add a sig label by:
(1) mentioning a sig: @kubernetes/sig-<team-name>-misc
(2) specifying the label manually: /sig <label>

Note: method (1) will trigger a notification to the team. You can find the team list here and label list here

@k8s-github-robot k8s-github-robot added the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Jun 8, 2017
@xiangpengzhao
Copy link
Contributor

/sig apps

@k8s-ci-robot k8s-ci-robot added the sig/apps Categorizes an issue or PR as relevant to SIG Apps. label Jun 9, 2017
@k8s-github-robot k8s-github-robot removed the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Jun 9, 2017
@xingzhou
Copy link
Contributor

xingzhou commented Jun 9, 2017

Prefer the second option as it's looks more nature to me

@0xmichalis
Copy link
Contributor

@soltysh @kubernetes/sig-apps-feature-requests

@iterion
Copy link
Contributor Author

iterion commented Jun 9, 2017

I also like the second form, but it seems that the easiest path to implementation would be to strip that time off and use it separately. Also, an argument could be made, that, since we're using the "standard" cron template we shouldn't mess with that.

It seems like this should be straightforward to implement:
Currently, we just pass time.Now() into the syncOne loop. The cron lib that k8s is using seems to work by just using times in the zone you care about. So, we could pull the timezone from the CronJob and pass that time with zone into the syncOne func.

//init the loc, needs error handling
//perhaps default to local if Timezone was invalid
loc, _ := time.LoadLocation(cronjob.Spec.Timezone)

//set timezone,  
now := time.Now().In(loc)

Also, I'm happy to try to implement this.

@mml
Copy link
Contributor

mml commented Jun 9, 2017

I'm not the right person to review your PR, but I do have a few questions about semantics.

  • What are the semantics when no timezone is specified?
  • Does the implementation depend on the timezone being correctly configured on the controller host?
  • When do we validate timezone legality? At object create/update time, or later?
  • What happens if a timezone name was legal when the API object was created, but in the future no longer exists? This could happen because a name is removed (maybe a city or nation ceases to exist), or because the underlying tzdata package has changed to an older version.

@iterion
Copy link
Contributor Author

iterion commented Jun 9, 2017

If no timezone is defined it would default to the current behavior which would be to use the timezone of the host. I would guess in most cases this would be configured as UTC, but a user could really choose any timezone. As such, there is no greater dependency on timezone being correctly configured on the host as a result of this PR.

I currently just bail out and use the host's timezone if the user specified something invalid. Here, I'm leaning on golang's time.LoadLocation(str) method. Any string accepted by that function would be considered valid. I'm not currently validating this at create/update time, though it seems that would be a pretty easy thing to do.

As far as the last question, that one's a bit more undefined. Currently, and probably in the future, it would most likely fall back to scheduling the job based on the default time zone. I'm open to suggestions on how to handle this case.

@soltysh soltysh self-assigned this Aug 2, 2017
@soltysh soltysh added the kind/feature Categorizes issue or PR as related to a new feature. label Aug 2, 2017
@soltysh
Copy link
Contributor

soltysh commented Aug 2, 2017

What happens if a timezone name was legal when the API object was created, but in the future no longer exists? This could happen because a name is removed (maybe a city or nation ceases to exist), or because the underlying tzdata package has changed to an older version.

My suggestion for that would be to fail job creation with an appropriate event. But frankly speaking I doubt this happens that often. I'll make sure to have this added in your implementation @iterion.

@whybeyoung
Copy link

and cron format not support year filed, how to run a job only once at a specified time?

@c9s
Copy link

c9s commented Dec 2, 2017

👍

@mattfarina
Copy link
Contributor

@iterion Thanks for the issue and pull request. We discussed it quite a bit over the past week, both in SIG Apps and in SIG Architecture. While the use cases this could be useful for make sense we also had to discuss the downsides. For example, every compliant Kubernetes distribution would need to have a timezone database and keep it up to date. Timezones, daylight savings time, and the details of making this work while the real world details can and do shift is hard. This would be additional work for the more than 20 distributions that exist today.

At the same time there has been a shift in Kubernetes development. I'm not sure how widely communicated it has been (the communication was another point discussed in SIG Architecture). The shift is a desire to have core Kubernetes focus on stability and being slow moving. People are now encouraged to innovate and solve these kinds of problems in the ecosystem rather than core.

Instead of putting this in Kubernetes the ask is to:

  1. Develop this in the ecosystem (e.g., a controller) that others can use. Distribute it, solve the problems there, and see what update looks like
  2. If the solution is widely adopted and can be used by everyone (including small scale, multi-cluster, etc) then it could be considered for core Kubernetes

If it helps, this isn't the only features that's receiving this same direction (even in the same SIG Architecture meeting). I'm starting to think of this in a similar vein to something like wordpress (I use wordpress because it's a very common example). There difference between it being plugin or part of wordpress itself. The current direction is to encourage innovation in plugins, so to speak.

If you build something like this we'd like to have it demo'd at SIG Apps to showcase it for folks. It could also be demo'd at the community meeting.

If you have questions on this please feel free to reach out to me.

@soltysh
Copy link
Contributor

soltysh commented Feb 5, 2018

Given the previous comment I feel like we can close this issue. Feel free to reopen if you don't agree.

@soltysh soltysh closed this as completed Feb 5, 2018
@mcluseau
Copy link
Contributor

FWIW, "how to set the timezone" is asked by every single of my users that makes a CronJob (developers and admins alike).

@benlangfeld
Copy link

benlangfeld commented Jun 26, 2018

@iterion What did you end up doing about this? We're considering various options for our use case, including

  • running from a fork of kubernetes with your fix
  • Copying the cron controller code into an external controller, and using a different apiVersion to distinguish the two.
  • Building something specific to our project in Ruby which mirrors the kube cronjob controller implementation but based on an app-specific config file rather than CronJob resources.

If you already have something that's working for you that we could use as a starting point, or if we could collaborate on something that could serve to encourage @mattfarina to reverse his decision not to permit including this (which dramatically harms the adoption of this kube feature), that would be awesome.

@iterion
Copy link
Contributor Author

iterion commented Jun 26, 2018

I've actually switched to a new job, so the original problem doesn't exist for me, personally, anymore. As far as I know, my former colleagues are just dealing with the pain that results from it.

That said, if I had the time to devote to this I would likely just write a small controller that would watch and modify CronJobs as needed. I'd likely just add an annotation to CronJobs with the desired time and time zone and have the controller cast that schedule to UTC and modify the CronJobs. You could just run this sync loop at an interval to catch the problem edges, i.e. when daylight savings starts or ends. In this way, the controller would just be a very slim wrapper on top of the already existing CronJob API.

@soltysh
Copy link
Contributor

soltysh commented Jun 27, 2018

encourage @mattfarina to reverse his decision not to permit including this

To make it clear, this decision was done collectively during one of sig-architecture meetings. Matt was only the person expressing that decision in this issue.

@bronger
Copy link

bronger commented Jul 30, 2018

The problem of timezones with all bells and whistles has long been solved in the Linux world. Why is this that much of a prerequisite for a Kubernetes deployment?

It is difficult to discuss about the significance of use cases I admit. However (and in my humble opinion), being able to schedule services in sync with the services' users seems important enough to me to be eligible for Kubernetes core.

@joekohlsdorf
Copy link

joekohlsdorf commented Aug 31, 2018

Would anyone be willing to write an operator with a TimezoneAwareCronjobController? Most of the code written by @iterion can probably be recycled.
Moving TZ handling into a packaged application would solve the distribution argument that killed first class cronjob support in Kubernetes.

@tcolgate
Copy link

tcolgate commented Oct 5, 2018

FWIW, https://gopkg.in/robfig/cron.v2 (which the existing controller uses am older subset of), does support passing in TZ, and a slight richer set of time definitions.
We have a controller internally that creates cron jobs for some of our internal workloads, and it is an outstanding feature request to be able to specifcy timezone for those. I need to work out how to safely update an existing cronjob definition without accidentally skipping, or rerunning jobs. At this point it feels like it might actually be easier to steal the job scheduling from the exiting contoller and create job definitions directly (rather than cronjobs).

@gzzo
Copy link

gzzo commented Nov 5, 2018

Since I've started using K8s I come back to this issue twice a year, when I'm forced to change the schedule on all the CronJobs, hoping that it's finally been reopened. If chronos and crontab have the feature, how is K8s still not supporting this?

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 26, 2022
@command-tab
Copy link

Not stale

@jsalatiel
Copy link

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 26, 2022
@sftim
Copy link
Contributor

sftim commented Jun 26, 2022

We could close this in favor of KEP 3140.

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 24, 2022
@command-tab
Copy link

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 24, 2022
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 23, 2022
@command-tab
Copy link

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 23, 2022
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 24, 2023
@onedr0p
Copy link

onedr0p commented Mar 24, 2023

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 24, 2023
@sergeyshevch
Copy link
Member

Maybe we should close it or set it as lifecycle frozen?

@MrSeccubus
Copy link

Given the amount of comments users have given this or the times somebody has set /remote lifecycle-stale I uess you should all view this as upvotes of users saying that this is a serious issues that the community is incapable of adressing now for almost 6 years.

@dims
Copy link
Member

dims commented Mar 27, 2023

FYI / FWIW @MrSeccubus

@liggitt
Copy link
Member

liggitt commented May 24, 2023

time-zone support in cronjobs is GA in 1.27

/close

@k8s-ci-robot
Copy link
Contributor

@liggitt: Closing this issue.

In response to this:

time-zone support in cronjobs is GA in 1.27

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@MkrierPharmanity
Copy link

its has been 6 years, do kubernetes finally support tz in cronjob ?

@dims
Copy link
Member

dims commented Jun 2, 2023

its has been 6 years, do kubernetes finally support tz in cronjob ?

https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#time-zones

@cprivitere
Copy link
Member

time-zone support in cronjobs is GA in 1.27

/close

kubernetes do finally support tz in cronjob

@cprivitere
Copy link
Member

/pony

@k8s-ci-robot
Copy link
Contributor

@cprivitere: pony image

In response to this:

/pony

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/batch area/workload-api/cronjob kind/feature Categorizes issue or PR as related to a new feature. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. sig/apps Categorizes an issue or PR as relevant to SIG Apps.
Projects
None yet