Skip to content

WIP: Remove the "100 missed start times" CronJob limit - #89397

Closed
vllry wants to merge 1 commit into
kubernetes:masterfrom
vllry:cj-100
Closed

vllry wants to merge 1 commit into
kubernetes:masterfrom
vllry:cj-100

Conversation

@vllry

@vllry vllry commented Mar 24, 2020

Copy link
Copy Markdown
Contributor

(This code was written as a patch, and needs some cleanup to directly solve the problem in k/k).

Removes the 100 missed schedule times limit from CronJobs.

Introduces a binary search approach with the schedule.Next() function in the existing 3rd party cron module. The binary search provides a much sharper upper bound on the timespan checked, and number of Next() calls. EG, a minutely CronJob that has been failing for a year requires checking only ~20 Next() calls (log2(60*24*365) + 1) in order to get the most recent missed schedule.

Co-Authored-By: Tom Wanielista tomwans@users.noreply.github.com
Co-Authored-By: Kevin Yang 16491459+yangkev@users.noreply.github.com

What type of PR is this?
/kind bug

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Removes a well-known limitation, where CronJobs could only miss 100 consecutive start times before needing to be re-created.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:
Needs doc updates, TBD.


Co-Authored-By: Tom Wanielista <tomwans@users.noreply.github.com>

Co-Authored-By: Kevin Yang <16491459+yangkev@users.noreply.github.com>
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/bug Categorizes issue or PR as related to a bug. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. and removed needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels Mar 24, 2020
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: vllry
To complete the pull request process, please assign erictune
You can assign the PR to them by writing /assign @erictune in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added sig/apps Categorizes an issue or PR as relevant to SIG Apps. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Mar 24, 2020
@vllry

vllry commented Mar 24, 2020

Copy link
Copy Markdown
Contributor Author

/uncc @soltysh
/uncc @smarterclayton

/test pull-kubernetes-conformance-kind-ga-only-parallel

@vllry

vllry commented Mar 24, 2020

Copy link
Copy Markdown
Contributor Author

/test pull-kubernetes-e2e-kind

@k8s-ci-robot

k8s-ci-robot commented Mar 24, 2020 •

Copy link
Copy Markdown
Contributor

@vllry: The following test failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
pull-kubernetes-verify d4dcc26 link /test pull-kubernetes-verify

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@fejta-bot

Copy link
Copy Markdown

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 22, 2020
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

@vllry: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 22, 2020
@fejta-bot

Copy link
Copy Markdown

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jul 22, 2020
@fejta-bot

Copy link
Copy Markdown

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

@k8s-ci-robot

Copy link
Copy Markdown
Contributor

@fejta-bot: Closed this PR.

Details

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@alexey-gavrilov-flant

Copy link
Copy Markdown
Contributor

/reopen

@k8s-ci-robot

Copy link
Copy Markdown
Contributor

@alexey-gavrilov-flant: You can't reopen an issue/PR unless you authored it or you are a collaborator.

Details

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ltagliamonte-dd

Copy link
Copy Markdown
Contributor

@vllry is there any reason why this PR didn't get attention?
Few people are "suffering" from:

Cannot determine if job needs to be started: too many missed start time (> 100). Set or decrease .spec.startingDeadlineSeconds or check clock skew

@soltysh

soltysh commented Nov 26, 2020

Copy link
Copy Markdown
Contributor

There's a new cronjob controller being written see #93370

@2rs2ts

2rs2ts commented Dec 2, 2020

Copy link
Copy Markdown
Contributor

@soltysh The new cronjob controller has the same problem, according to this comment #81557 (comment)

I think this PR might be redundant with that one, #81557, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/bug Categorizes issue or PR as related to a bug. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/apps Categorizes an issue or PR as relevant to SIG Apps. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants