New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ScheduledJob controller proposal #11980

Merged
merged 1 commit into from Jan 25, 2016

Conversation

Projects
None yet

@googlebot googlebot added the cla: yes label Jul 29, 2015

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Jul 29, 2015

Can one of the admins verify that this patch is reasonable to test? (reply "ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain to ixdy.

k8s-bot commented Jul 29, 2015

Can one of the admins verify that this patch is reasonable to test? (reply "ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain to ixdy.

@@ -0,0 +1,132 @@
# ScheduledJob Controller

This comment has been minimized.

@timothysc

timothysc Jul 31, 2015

Member

Comment still holds on name

@timothysc

timothysc Jul 31, 2015

Member

Comment still holds on name

This comment has been minimized.

@smarterclayton

smarterclayton Jul 31, 2015

Contributor

The problem with "Cron" is that is technically focused (a particular linux
tool) but is opaque to higher level users who come from different
backgrounds.

Even the Chronos GitHub page describes it with "It allows you to schedule
your jobs using ISO8601 repeating interval notation, which enables more
flexibility in job scheduling". RepeatingJob or CalendarJob or
RecurringJob all seem less clear to an outside user (someone who is not an
expert at these systems) than ScheduledJob.

On Fri, Jul 31, 2015 at 12:41 PM, Timothy St. Clair <
notifications@github.com> wrote:

In docs/proposals/scheduledjob.md
#11980 (comment)
:

@@ -0,0 +1,132 @@
+# ScheduledJob Controller

Comment still holds on name


Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/pull/11980/files#r35991090
.

Clayton Coleman | Lead Engineer, OpenShift

@smarterclayton

smarterclayton Jul 31, 2015

Contributor

The problem with "Cron" is that is technically focused (a particular linux
tool) but is opaque to higher level users who come from different
backgrounds.

Even the Chronos GitHub page describes it with "It allows you to schedule
your jobs using ISO8601 repeating interval notation, which enables more
flexibility in job scheduling". RepeatingJob or CalendarJob or
RecurringJob all seem less clear to an outside user (someone who is not an
expert at these systems) than ScheduledJob.

On Fri, Jul 31, 2015 at 12:41 PM, Timothy St. Clair <
notifications@github.com> wrote:

In docs/proposals/scheduledjob.md
#11980 (comment)
:

@@ -0,0 +1,132 @@
+# ScheduledJob Controller

Comment still holds on name


Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/pull/11980/files#r35991090
.

Clayton Coleman | Lead Engineer, OpenShift

This comment has been minimized.

@timothysc

timothysc Jul 31, 2015

Member

Proposal was around (diurnal, periodic, etc.) There may be a couple more.

@timothysc

timothysc Jul 31, 2015

Member

Proposal was around (diurnal, periodic, etc.) There may be a couple more.

This comment has been minimized.

@smarterclayton

smarterclayton Jul 31, 2015

Contributor

Periodic suffers some of the same usability concerns. I do think API
objects should be precise in their meaning, but designed for end users, not
systems researchers.

On Fri, Jul 31, 2015 at 1:02 PM, Timothy St. Clair <notifications@github.com

wrote:

In docs/proposals/scheduledjob.md
#11980 (comment)
:

@@ -0,0 +1,132 @@
+# ScheduledJob Controller

Proposal was around (diurnal, periodic, etc.) There may be a couple more.


Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/pull/11980/files#r35992880
.

Clayton Coleman | Lead Engineer, OpenShift

@smarterclayton

smarterclayton Jul 31, 2015

Contributor

Periodic suffers some of the same usability concerns. I do think API
objects should be precise in their meaning, but designed for end users, not
systems researchers.

On Fri, Jul 31, 2015 at 1:02 PM, Timothy St. Clair <notifications@github.com

wrote:

In docs/proposals/scheduledjob.md
#11980 (comment)
:

@@ -0,0 +1,132 @@
+# ScheduledJob Controller

Proposal was around (diurnal, periodic, etc.) There may be a couple more.


Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/pull/11980/files#r35992880
.

Clayton Coleman | Lead Engineer, OpenShift

This comment has been minimized.

@bgrant0607

bgrant0607 Aug 11, 2015

Member

I'm fine with ScheduledJob. It's similar to ScheduledExecutorService in Java.

@bgrant0607

bgrant0607 Aug 11, 2015

Member

I'm fine with ScheduledJob. It's similar to ScheduledExecutorService in Java.

This comment has been minimized.

@bgrant0607

bgrant0607 Nov 12, 2015

Member

Could also call it CalendaredJob

@bgrant0607

bgrant0607 Nov 12, 2015

Member

Could also call it CalendaredJob

@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Aug 5, 2015

Member

When you implement this please put in expapi. (#12001).

Member

erictune commented Aug 5, 2015

When you implement this please put in expapi. (#12001).

@soltysh

This comment has been minimized.

Show comment
Hide comment
@soltysh

soltysh Aug 11, 2015

Contributor

@erictune gotcha.

Contributor

soltysh commented Aug 11, 2015

@erictune gotcha.

Show outdated Hide outdated docs/proposals/scheduledjob.md
type ScheduledJobStatus struct {
// PendingExecutions are the job runs pending for execution
PendingExecutions []Job

This comment has been minimized.

@bgrant0607

bgrant0607 Aug 11, 2015

Member

Much of the same feedback applies to this API as to Job.

@bgrant0607

bgrant0607 Aug 11, 2015

Member

Much of the same feedback applies to this API as to Job.

This comment has been minimized.

@soltysh

soltysh Aug 11, 2015

Contributor

Yeah, I'll address them as soon as we have consensus on Job API, which I'm hoping will happen withing next couple of days :)

@soltysh

soltysh Aug 11, 2015

Contributor

Yeah, I'll address them as soon as we have consensus on Job API, which I'm hoping will happen withing next couple of days :)

This comment has been minimized.

@pmorie

pmorie Aug 17, 2015

Member

My feedback re: Job API and overall status applies here too.

@pmorie

pmorie Aug 17, 2015

Member

My feedback re: Job API and overall status applies here too.

@soltysh

This comment has been minimized.

Show comment
Hide comment
@soltysh

soltysh Aug 18, 2015

Contributor

@bgrant0607 @erictune I've updated the proposal, cutting the status structure significantly. I've only left there a list of currently executing jobs, which is needed to track these executions and fulfill the defined ConcurrentPolicy.

Contributor

soltysh commented Aug 18, 2015

@bgrant0607 @erictune I've updated the proposal, cutting the status structure significantly. I've only left there a list of currently executing jobs, which is needed to track these executions and fulfill the defined ConcurrentPolicy.

@soltysh

This comment has been minimized.

Show comment
Hide comment
Contributor

soltysh commented Aug 18, 2015

Show outdated Hide outdated docs/proposals/scheduledjob.md
type ScheduledJobStatus struct {
// Jobs are the currently running instances of a ScheduledJob.
Jobs []Job

This comment has been minimized.

@mikedanese

mikedanese Aug 18, 2015

Member

Why not a selector query? (you'd have to add a selector to ScheduledJobSpec)

@mikedanese

mikedanese Aug 18, 2015

Member

Why not a selector query? (you'd have to add a selector to ScheduledJobSpec)

This comment has been minimized.

@soltysh

soltysh Aug 18, 2015

Contributor

How would you differentiate concurrent runs? We'd have to generate some random uuid for that selector in that case.

@soltysh

soltysh Aug 18, 2015

Contributor

How would you differentiate concurrent runs? We'd have to generate some random uuid for that selector in that case.

This comment has been minimized.

@mikedanese

mikedanese Aug 18, 2015

Member

The selector should select all Jobs created by the ScheduledJob. We can sort them by CreationTimestamp. What do you mean differentiate? How does having an array help you differentiate?

@mikedanese

mikedanese Aug 18, 2015

Member

The selector should select all Jobs created by the ScheduledJob. We can sort them by CreationTimestamp. What do you mean differentiate? How does having an array help you differentiate?

This comment has been minimized.

@soltysh

soltysh Aug 18, 2015

Contributor

Yeah, I haven't thought about sorting them by CreationTimestamp, which generally speaking allows me to distinguish (that's the word I've meant to use, sorry :/) different ScheduledJob executions/runs needed to properly handle ConcurrentPolicy.

@soltysh

soltysh Aug 18, 2015

Contributor

Yeah, I haven't thought about sorting them by CreationTimestamp, which generally speaking allows me to distinguish (that's the word I've meant to use, sorry :/) different ScheduledJob executions/runs needed to properly handle ConcurrentPolicy.

This comment has been minimized.

@bgrant0607

bgrant0607 Aug 25, 2015

Member

Sorting by CreationTimestamp seems exactly what the user would want, since ScheduledJobs are launched based on time.

Please no array.

@bgrant0607

bgrant0607 Aug 25, 2015

Member

Sorting by CreationTimestamp seems exactly what the user would want, since ScheduledJobs are launched based on time.

Please no array.

This comment has been minimized.

@soltysh

soltysh Aug 25, 2015

Contributor

Array removed, added selector to ScheduledJobSpec.

@soltysh

soltysh Aug 25, 2015

Contributor

Array removed, added selector to ScheduledJobSpec.

@davidopp davidopp assigned mikedanese and unassigned davidopp Aug 18, 2015

Show outdated Hide outdated docs/proposals/scheduledjob.md
Jobs []Job
// Successful tracks the overall amount of successful completions of this job.
Successful int

This comment has been minimized.

@bgrant0607

bgrant0607 Aug 25, 2015

Member

If these accumulate indefinitely, they should probably be int64.

@bgrant0607

bgrant0607 Aug 25, 2015

Member

If these accumulate indefinitely, they should probably be int64.

This comment has been minimized.

@soltysh

soltysh Aug 25, 2015

Contributor

Done.

@soltysh

soltysh Aug 25, 2015

Contributor

Done.

Show outdated Hide outdated docs/proposals/scheduledjob.md
// SkipNextConcurrent forbids concurrent runs, skipping next run if previous
// hasn't finished yet.
SkipNextConcurrent ConcurrentPolicy = "SkipNextConcurrent"

This comment has been minimized.

@bgrant0607

bgrant0607 Aug 25, 2015

Member

You don't need to include "Concurrent" in all the values. See, for instance, RestartPolicy.

Also, I'd call this ConcurrencyPolicy.

@bgrant0607

bgrant0607 Aug 25, 2015

Member

You don't need to include "Concurrent" in all the values. See, for instance, RestartPolicy.

Also, I'd call this ConcurrencyPolicy.

This comment has been minimized.

@soltysh

soltysh Aug 25, 2015

Contributor

Done.

@soltysh

soltysh Aug 25, 2015

Contributor

Done.

Show outdated Hide outdated docs/proposals/scheduledjob.md
Schedule string
// ConcurrentPolicy specifies how to treat concurrent executions of a Job.
ConcurrentPolicy ConcurrentPolicy

This comment has been minimized.

@bgrant0607

bgrant0607 Aug 25, 2015

Member

Is there a default value?

@bgrant0607

bgrant0607 Aug 25, 2015

Member

Is there a default value?

This comment has been minimized.

@soltysh

soltysh Aug 25, 2015

Contributor

Yes, it's AllowConcurrent. I've put it on the type declaration, similarly like it's done with RestartPolicy.

@soltysh

soltysh Aug 25, 2015

Contributor

Yes, it's AllowConcurrent. I've put it on the type declaration, similarly like it's done with RestartPolicy.

Show outdated Hide outdated docs/proposals/scheduledjob.md
ConcurrentPolicy ConcurrentPolicy
// BlockOnFailure suspends scheduling of next job runs after a failed one.
BlockOnFailure bool

This comment has been minimized.

@bgrant0607

bgrant0607 Aug 25, 2015

Member

Is there a default value?

@bgrant0607

bgrant0607 Aug 25, 2015

Member

Is there a default value?

This comment has been minimized.

@soltysh

soltysh Aug 25, 2015

Contributor

It's false, I'll update the description.

@soltysh

soltysh Aug 25, 2015

Contributor

It's false, I'll update the description.

Show outdated Hide outdated docs/proposals/scheduledjob.md
* repeatedly at a specified point in time.
There is already a discussion regarding that particular subject:
* Distributed CRON jobs [#2156](https://github.com/GoogleCloudPlatform/kubernetes/issues/2156)

This comment has been minimized.

@bgrant0607

bgrant0607 Aug 25, 2015

Member

Repo has moved. Use the redirector:
http://issues.k8s.io/2156

@bgrant0607

bgrant0607 Aug 25, 2015

Member

Repo has moved. Use the redirector:
http://issues.k8s.io/2156

This comment has been minimized.

@soltysh

soltysh Aug 25, 2015

Contributor

Fixed.

@soltysh

soltysh Aug 25, 2015

Contributor

Fixed.

Show outdated Hide outdated docs/proposals/scheduledjob.md
type ScheduledJobSpec struct {
// JobSpec is a structure defining the expected behavior of a job.
JobSpec JobSpec

This comment has been minimized.

This comment has been minimized.

@soltysh

soltysh Aug 25, 2015

Contributor

After talking to @smarterclayton I've renamed JobSpec to be pointer to JobSpec.

@soltysh

soltysh Aug 25, 2015

Contributor

After talking to @smarterclayton I've renamed JobSpec to be pointer to JobSpec.

@soltysh

This comment has been minimized.

Show comment
Hide comment
@soltysh

soltysh Aug 25, 2015

Contributor

@mikedanese @bgrant0607 addressed your comments, appreciate another round of reviews.

Contributor

soltysh commented Aug 25, 2015

@mikedanese @bgrant0607 addressed your comments, appreciate another round of reviews.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment

k8s-bot commented Aug 25, 2015

GCE e2e build/test passed for commit e234d1f.

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Aug 27, 2015

Collaborator

Labelling this PR as size/L

Collaborator

k8s-merge-robot commented Aug 27, 2015

Labelling this PR as size/L

@mikedanese

This comment has been minimized.

Show comment
Hide comment
@mikedanese

mikedanese Sep 17, 2015

Member

Let's continue this review targeting v1.2

Member

mikedanese commented Sep 17, 2015

Let's continue this review targeting v1.2

@erictune erictune added this to the v1.2 milestone Sep 22, 2015

Show outdated Hide outdated docs/proposals/scheduledjob.md
// - 2015-07-21T14:00:00Z - represents date and time in UTC
// - R/2015-07-21T14:00:00Z/P1D - represents endlessly repeating interval (1 day), starting from given date
Schedule string

This comment has been minimized.

@sdminonne

sdminonne Sep 23, 2015

Contributor

@soltysh my understanding here is that ISO8601 format it's going to be a necessary pain but as far as I understood string are not mergeable and to me schedule spec looks something we may want to update quite often. Should we consider a more structured representation instead?

@sdminonne

sdminonne Sep 23, 2015

Contributor

@soltysh my understanding here is that ISO8601 format it's going to be a necessary pain but as far as I understood string are not mergeable and to me schedule spec looks something we may want to update quite often. Should we consider a more structured representation instead?

This comment has been minimized.

@soltysh

soltysh Sep 23, 2015

Contributor

My understanding is that you won't be merging schedule, but rather replacing an old one with a new one. Although ISO8601 format seems clunky at first, it's very readable and easily customizable. Using any kind of structured representation would introduce unnecessary complication imho.

@soltysh

soltysh Sep 23, 2015

Contributor

My understanding is that you won't be merging schedule, but rather replacing an old one with a new one. Although ISO8601 format seems clunky at first, it's very readable and easily customizable. Using any kind of structured representation would introduce unnecessary complication imho.

This comment has been minimized.

@sdminonne

sdminonne Sep 23, 2015

Contributor

I was not discussing ISO8601 here but the string representation.
I'm not sure I fully understand all the implications of replacing schedules. Do you plan to delete/create? In any cases may you add something about it in the doc (except if I missed something)?

@sdminonne

sdminonne Sep 23, 2015

Contributor

I was not discussing ISO8601 here but the string representation.
I'm not sure I fully understand all the implications of replacing schedules. Do you plan to delete/create? In any cases may you add something about it in the doc (except if I missed something)?

This comment has been minimized.

@soltysh

soltysh Sep 23, 2015

Contributor

My general idea (which as you point out needs clear statement somewhere in this proposal) is that any changes you'll do to the scheduledjob will reflect in every new run, it does not affect nor delete existing executions. Eg. I have a job running every 15 mins, it's 1.45, so one is currently running, and I change the schedule to be running every hour, that means the existing job will finish what it started and next one will start at 2, then at 3 etc. @sdminonne does it make sense?

@soltysh

soltysh Sep 23, 2015

Contributor

My general idea (which as you point out needs clear statement somewhere in this proposal) is that any changes you'll do to the scheduledjob will reflect in every new run, it does not affect nor delete existing executions. Eg. I have a job running every 15 mins, it's 1.45, so one is currently running, and I change the schedule to be running every hour, that means the existing job will finish what it started and next one will start at 2, then at 3 etc. @sdminonne does it make sense?

This comment has been minimized.

@sdminonne

sdminonne Sep 23, 2015

Contributor

@soltysh absolutely but when you say I change the schedule to be running every hour are you updating the scheduleJob spec or are you creating another one? My understanding is that we should simply updating the current scheduleJob. But to do this we need the iso8601 spec to be mergeable. Am I missing something?

@sdminonne

sdminonne Sep 23, 2015

Contributor

@soltysh absolutely but when you say I change the schedule to be running every hour are you updating the scheduleJob spec or are you creating another one? My understanding is that we should simply updating the current scheduleJob. But to do this we need the iso8601 spec to be mergeable. Am I missing something?

This comment has been minimized.

@mikedanese

mikedanese Sep 23, 2015

Member

I don't see much utility in partially updating a schedule over replacing a schedule.

@mikedanese

mikedanese Sep 23, 2015

Member

I don't see much utility in partially updating a schedule over replacing a schedule.

This comment has been minimized.

@soltysh

soltysh Sep 24, 2015

Contributor

Right, those where my feelings as well, though I know nothing about mergeable strings and need to read about it a bit. But again, when updating you'll just modify the current schedule so that it matches your new expectations and submit the new one which will replace the old one.

@soltysh

soltysh Sep 24, 2015

Contributor

Right, those where my feelings as well, though I know nothing about mergeable strings and need to read about it a bit. But again, when updating you'll just modify the current schedule so that it matches your new expectations and submit the new one which will replace the old one.

This comment has been minimized.

@bgrant0607

bgrant0607 Nov 12, 2015

Member

Maybe an example would help.

For instance, one might change R/2015-07-21T14:00:00Z/P1D to R/2015-07-26T20:00:00Z/P1D.

@sdminonne Do you want to be able to independently change the start date, start time, and periodicity?

@bgrant0607

bgrant0607 Nov 12, 2015

Member

Maybe an example would help.

For instance, one might change R/2015-07-21T14:00:00Z/P1D to R/2015-07-26T20:00:00Z/P1D.

@sdminonne Do you want to be able to independently change the start date, start time, and periodicity?

This comment has been minimized.

@sdminonne

sdminonne Nov 14, 2015

Contributor

@bgrant0607, doesn't matter replacing completely the scheduler is good enough for us. Thanks

@sdminonne

sdminonne Nov 14, 2015

Contributor

@bgrant0607, doesn't matter replacing completely the scheduler is good enough for us. Thanks

@mikedanese

This comment has been minimized.

Show comment
Hide comment
@mikedanese

mikedanese Sep 23, 2015

Member

@soltysh are you going to work on this for v1.2?

Edit: oops just saw the milestone.

Member

mikedanese commented Sep 23, 2015

@soltysh are you going to work on this for v1.2?

Edit: oops just saw the milestone.

@soltysh

This comment has been minimized.

Show comment
Hide comment
@soltysh

soltysh Sep 24, 2015

Contributor

@mikedanese yes sir, I'm waiting for reviews but I guess those will happen when 1.1 will be closed. You're welcome to comment at any time, though.

Contributor

soltysh commented Sep 24, 2015

@mikedanese yes sir, I'm waiting for reviews but I guess those will happen when 1.1 will be closed. You're welcome to comment at any time, though.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Sep 30, 2015

Unit, integration and GCE e2e build/test failed for commit e234d1f.

k8s-bot commented Sep 30, 2015

Unit, integration and GCE e2e build/test failed for commit e234d1f.

Show outdated Hide outdated docs/proposals/scheduledjob.md
// BlockOnFailure suspends scheduling of next job runs after a failed one,
// by default it is false.
BlockOnFailure bool

This comment has been minimized.

@mikedanese

mikedanese Nov 3, 2015

Member

Isn't this just a type of concurrency policy?

@mikedanese

mikedanese Nov 3, 2015

Member

Isn't this just a type of concurrency policy?

This comment has been minimized.

@mikedanese

mikedanese Nov 3, 2015

Member

Misread the doc. What is a failed run?

@mikedanese

mikedanese Nov 3, 2015

Member

Misread the doc. What is a failed run?

This comment has been minimized.

@soltysh

soltysh Nov 4, 2015

Contributor

Hmmm... with current Jobs API I'll probably have to introduce some kind of succeeded/failed pod rate that will allow deciding what is a failed run, WDYT?

@soltysh

soltysh Nov 4, 2015

Contributor

Hmmm... with current Jobs API I'll probably have to introduce some kind of succeeded/failed pod rate that will allow deciding what is a failed run, WDYT?

This comment has been minimized.

@mikedanese

mikedanese Nov 4, 2015

Member

Sounds good. Should we remove from proposal until that is implemented?

@mikedanese

mikedanese Nov 4, 2015

Member

Sounds good. Should we remove from proposal until that is implemented?

This comment has been minimized.

@soltysh

soltysh Nov 5, 2015

Contributor

I'm not sure if that should be part of a job, though. Initially I was thinking this would be only applicable here, but thinking about it more it would benefit jobs actually. @erictune wdyt? This should be pretty easy to implement actually.

@soltysh

soltysh Nov 5, 2015

Contributor

I'm not sure if that should be part of a job, though. Initially I was thinking this would be only applicable here, but thinking about it more it would benefit jobs actually. @erictune wdyt? This should be pretty easy to implement actually.

This comment has been minimized.

@soltysh

soltysh Nov 5, 2015

Contributor

@bgrant0607 ptal as well, regarding job API changes, or should I create a separate proposal for that?

@soltysh

soltysh Nov 5, 2015

Contributor

@bgrant0607 ptal as well, regarding job API changes, or should I create a separate proposal for that?

This comment has been minimized.

@bgrant0607

bgrant0607 Nov 12, 2015

Member

Is this thread about Concurrency and BlockOnFailure? Those don't make sense for Job.

@bgrant0607

bgrant0607 Nov 12, 2015

Member

Is this thread about Concurrency and BlockOnFailure? Those don't make sense for Job.

This comment has been minimized.

@soltysh

soltysh Nov 12, 2015

Contributor

It's about having an option (in jobs) which allows defining when a job is considered failed. Having thought about it more, though I think that it's client role to decide when a job succeeded or failed. In this case it's the ScheduledJob that is Job's client that needs that information to decided about job.

@soltysh

soltysh Nov 12, 2015

Contributor

It's about having an option (in jobs) which allows defining when a job is considered failed. Having thought about it more, though I think that it's client role to decide when a job succeeded or failed. In this case it's the ScheduledJob that is Job's client that needs that information to decided about job.

@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Jan 9, 2016

Member

Regarding whether we need this or let people figure it out:
Thousands of teams in Google use something along these lines. We should make it easy.

Member

bgrant0607 commented Jan 9, 2016

Regarding whether we need this or let people figure it out:
Thousands of teams in Google use something along these lines. We should make it easy.

Show outdated Hide outdated docs/proposals/scheduledjob.md
}
```
+### Running ScheduledJobs using kubectl

This comment has been minimized.

@xiang90

xiang90 Jan 9, 2016

Contributor

broken markdown. remove the extra + at the beginning.

@xiang90

xiang90 Jan 9, 2016

Contributor

broken markdown. remove the extra + at the beginning.

@soltysh

This comment has been minimized.

Show comment
Hide comment
@soltysh

soltysh Jan 13, 2016

Contributor

Addressed last comment.
@erictune mind taking a look?

Contributor

soltysh commented Jan 13, 2016

Addressed last comment.
@erictune mind taking a look?

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Jan 13, 2016

GCE e2e test build/test passed for commit b00be17.

k8s-bot commented Jan 13, 2016

GCE e2e test build/test passed for commit b00be17.

@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Jan 13, 2016

Member

@bgrant0607
Agree that we need templates at some point. I think there is value in making progress on this and templates in parallel. So, I suggest the following evolution path for ScheduledJob:

  • Proceed with ScheduledJob as described here, except change JobTemplate JobSpec to be JobTemplate *JobSpec, thus allowing it to be optional in the future
  • Later, when Workflow is completed, add WorkflowTemplate *WorkflowSpec as an alternative to JobTemplate.
  • Even later, when arbitrary Templates are done, also add GenericTemplate *Template
  • Do not fret about it being called ScheduledJob even though it will be able to start things that are not Jobs. It can run jobs and still evokes the right idea in a user.
  • The spec may end up looking like this:
type ScheduledJobSpec {
....
// Specify exactly one of the following:
JobTemplate *JobSpec
WorkflowTemplate *WorkflowSpec 
GenericTemplate *Template
Member

erictune commented Jan 13, 2016

@bgrant0607
Agree that we need templates at some point. I think there is value in making progress on this and templates in parallel. So, I suggest the following evolution path for ScheduledJob:

  • Proceed with ScheduledJob as described here, except change JobTemplate JobSpec to be JobTemplate *JobSpec, thus allowing it to be optional in the future
  • Later, when Workflow is completed, add WorkflowTemplate *WorkflowSpec as an alternative to JobTemplate.
  • Even later, when arbitrary Templates are done, also add GenericTemplate *Template
  • Do not fret about it being called ScheduledJob even though it will be able to start things that are not Jobs. It can run jobs and still evokes the right idea in a user.
  • The spec may end up looking like this:
type ScheduledJobSpec {
....
// Specify exactly one of the following:
JobTemplate *JobSpec
WorkflowTemplate *WorkflowSpec 
GenericTemplate *Template
@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Jan 13, 2016

Member

@soltysh per previous comment, suggest change JobTemplate JobSpec to JobTemplate *JobSpec.

Member

erictune commented Jan 13, 2016

@soltysh per previous comment, suggest change JobTemplate JobSpec to JobTemplate *JobSpec.

@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Jan 13, 2016

Member

@erictune Your proposal for OneOf several template types sounds fine.

Was it a deliberate decision to not add Job metadata to the Job template?

Member

bgrant0607 commented Jan 13, 2016

@erictune Your proposal for OneOf several template types sounds fine.

Was it a deliberate decision to not add Job metadata to the Job template?

@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Jan 13, 2016

Member

No, that was not deliberate. IIUC, Brian is saying that we might want to make the following substitution:
JobTemplate *JobSpec ➡️ JobTemplate *JobTemplateSpec, where JobTemplateSpec is a new type that looks like this:

type JobTemplateSpec struct {
        // Standard object's metadata.
        // More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#metadata
        ObjectMeta `json:"metadata,omitempty"`

        // Specification of the desired behavior of the job.
        // More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#spec-and-status
        Spec JobSpec `json:"spec,omitempty"`
}

and likewise for WorkflowTemplate, should we ever get there.

I think we want to do it that way for consistency, and to allow users to name their created jobs.

Member

erictune commented Jan 13, 2016

No, that was not deliberate. IIUC, Brian is saying that we might want to make the following substitution:
JobTemplate *JobSpec ➡️ JobTemplate *JobTemplateSpec, where JobTemplateSpec is a new type that looks like this:

type JobTemplateSpec struct {
        // Standard object's metadata.
        // More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#metadata
        ObjectMeta `json:"metadata,omitempty"`

        // Specification of the desired behavior of the job.
        // More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#spec-and-status
        Spec JobSpec `json:"spec,omitempty"`
}

and likewise for WorkflowTemplate, should we ever get there.

I think we want to do it that way for consistency, and to allow users to name their created jobs.

@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Jan 13, 2016

Member

Okay, so summarizing steps to close out this PR:

  • remove pointer in slice
  • need pointer for JobTemplate
  • change type of JobTemplate from JobSpec to JobTemplateSpec, and define that type
  • move unique label/selector generation to be part of Job, and ScheduledJob to validate that its JobTemplates use this feature.
  • squash
  • LGTM
Member

erictune commented Jan 13, 2016

Okay, so summarizing steps to close out this PR:

  • remove pointer in slice
  • need pointer for JobTemplate
  • change type of JobTemplate from JobSpec to JobTemplateSpec, and define that type
  • move unique label/selector generation to be part of Job, and ScheduledJob to validate that its JobTemplates use this feature.
  • squash
  • LGTM
@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Jan 13, 2016

Member

@bgrant0607 wonder if you have an opinion on the ScheduledJobSpec.UniqueLabelKey approach in this PR, versus having the unique selector generation be part of Job itself.

Member

erictune commented Jan 13, 2016

@bgrant0607 wonder if you have an opinion on the ScheduledJobSpec.UniqueLabelKey approach in this PR, versus having the unique selector generation be part of Job itself.

@bgrant0607

This comment has been minimized.

Show comment
Hide comment
@bgrant0607

bgrant0607 Jan 13, 2016

Member

@erictune I'd prefer unique label/selector generation to be part of Job.

Member

bgrant0607 commented Jan 13, 2016

@erictune I'd prefer unique label/selector generation to be part of Job.

@soltysh

This comment has been minimized.

Show comment
Hide comment
@soltysh

soltysh Jan 14, 2016

Contributor

@erictune as requested:

  • removed pointer from Active
  • made JobTemplate pointer
  • made JobTemplate be JobTemplateSpec type
  • moved label generation to job (new section about JobSpec modifications)

I've also:

  • fixed all the ISO8601 leftovers
  • extended future evolution with proper links to workflow and templating

It's all squashed and waiting your sing-off.

Contributor

soltysh commented Jan 14, 2016

@erictune as requested:

  • removed pointer from Active
  • made JobTemplate pointer
  • made JobTemplate be JobTemplateSpec type
  • moved label generation to job (new section about JobSpec modifications)

I've also:

  • fixed all the ISO8601 leftovers
  • extended future evolution with proper links to workflow and templating

It's all squashed and waiting your sing-off.

@k8s-teamcity-mesosphere

This comment has been minimized.

Show comment
Hide comment
@k8s-teamcity-mesosphere

k8s-teamcity-mesosphere Jan 14, 2016

TeamCity OSS :: Kubernetes Mesos :: 4 - Smoke Tests Build 11550 outcome was SUCCESS
Summary: Tests passed: 1, ignored: 207 Build time: 00:03:52

TeamCity OSS :: Kubernetes Mesos :: 4 - Smoke Tests Build 11550 outcome was SUCCESS
Summary: Tests passed: 1, ignored: 207 Build time: 00:03:52

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot Jan 14, 2016

GCE e2e test build/test passed for commit 10de4c2.

k8s-bot commented Jan 14, 2016

GCE e2e test build/test passed for commit 10de4c2.

@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Jan 16, 2016

Member

This looks great. LGTM.

Member

erictune commented Jan 16, 2016

This looks great. LGTM.

@soltysh

This comment has been minimized.

Show comment
Hide comment
@soltysh

soltysh Jan 16, 2016

Contributor

Thanks for your help!

Contributor

soltysh commented Jan 16, 2016

Thanks for your help!

@alex-mohr

This comment has been minimized.

Show comment
Hide comment
@alex-mohr

alex-mohr Jan 21, 2016

Member

@k8s-bot unit test this

Member

alex-mohr commented Jan 21, 2016

@k8s-bot unit test this

@binarybana binarybana referenced this pull request Jan 21, 2016

Open

Investigate current K8S Scheduler landscape #1

0 of 4 tasks complete
@soltysh

This comment has been minimized.

Show comment
Hide comment
@soltysh

soltysh Jan 25, 2016

Contributor

@erictune any chance on having this one merged?

Contributor

soltysh commented Jan 25, 2016

@erictune any chance on having this one merged?

@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Jan 25, 2016

Member

Merging because this is just a new doc. Build failure is known flaky test.

Member

erictune commented Jan 25, 2016

Merging because this is just a new doc. Build failure is known flaky test.

erictune added a commit that referenced this pull request Jan 25, 2016

@erictune erictune merged commit 99f301d into kubernetes:master Jan 25, 2016

3 of 5 checks passed

Jenkins unit/integration 2574 tests run, 9 skipped, 0 failed.
Details
Submit Queue Github CI tests are not green.
Details
Jenkins GCE e2e 214 tests run, 98 skipped, 0 failed.
Details
cla/google All necessary CLAs are signed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Jan 25, 2016

Member

@soltysh merged.

Member

erictune commented Jan 25, 2016

@soltysh merged.

@soltysh

This comment has been minimized.

Show comment
Hide comment
@soltysh

soltysh Jan 25, 2016

Contributor

@erictune thank you!

Contributor

soltysh commented Jan 25, 2016

@erictune thank you!

@soltysh soltysh deleted the soltysh:scheduledjob_controller_proposal branch Jan 26, 2016

@yuvipanda

This comment has been minimized.

Show comment
Hide comment
@yuvipanda

yuvipanda Feb 27, 2016

Contributor

Is this being implemented anywhere?

Contributor

yuvipanda commented Feb 27, 2016

Is this being implemented anywhere?

@soltysh

This comment has been minimized.

Show comment
Hide comment
@soltysh

soltysh Feb 27, 2016

Contributor

Not yet. I'm planning to start working on it after 1.2 is released.
On Feb 27, 2016 3:25 AM, "Yuvi Panda" notifications@github.com wrote:

Is this being implemented anywhere?


Reply to this email directly or view it on GitHub
#11980 (comment)
.

Contributor

soltysh commented Feb 27, 2016

Not yet. I'm planning to start working on it after 1.2 is released.
On Feb 27, 2016 3:25 AM, "Yuvi Panda" notifications@github.com wrote:

Is this being implemented anywhere?


Reply to this email directly or view it on GitHub
#11980 (comment)
.

@haizaar

This comment has been minimized.

Show comment
Hide comment
@haizaar

haizaar Mar 25, 2016

Jobs guide says that "cron is expected in the next minor release". Do you plan this for 1.3 or 1.2.x?

haizaar commented Mar 25, 2016

Jobs guide says that "cron is expected in the next minor release". Do you plan this for 1.3 or 1.2.x?

@erictune

This comment has been minimized.

Show comment
Hide comment
@erictune

erictune Mar 25, 2016

Member

1.3

Member

erictune commented Mar 25, 2016

1.3

@zacharynevin

This comment has been minimized.

Show comment
Hide comment
@zacharynevin

zacharynevin Apr 26, 2016

What is the projected release date for v1.3?

What is the projected release date for v1.3?

@soltysh soltysh referenced this pull request Apr 29, 2016

Merged

Scheduledjob api #24970

@philips philips referenced this pull request May 18, 2016

Merged

ScheduledJob client #25475

@wernight

This comment has been minimized.

Show comment
Hide comment
@wernight

wernight Oct 11, 2016

Looks like documentation on http://kubernetes.io/docs/user-guide/jobs/#future-work didn't get updated (still pointing here). Should probably contain some of http://kubernetes.io/docs/user-guide/scheduled-jobs/

Also the reference documentation seems outdated http://kubernetes.io/docs/api-reference/batch/v1/definitions/

Last, but not least, the example on http://kubernetes.io/docs/user-guide/scheduled-jobs/#writing-a-scheduled-job-spec uses two non-standard notations: ? and 0/1 (not */1) which are not explained.

wernight commented Oct 11, 2016

Looks like documentation on http://kubernetes.io/docs/user-guide/jobs/#future-work didn't get updated (still pointing here). Should probably contain some of http://kubernetes.io/docs/user-guide/scheduled-jobs/

Also the reference documentation seems outdated http://kubernetes.io/docs/api-reference/batch/v1/definitions/

Last, but not least, the example on http://kubernetes.io/docs/user-guide/scheduled-jobs/#writing-a-scheduled-job-spec uses two non-standard notations: ? and 0/1 (not */1) which are not explained.

xingzhou pushed a commit to xingzhou/kubernetes that referenced this pull request Dec 15, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment