Support for ad hoc scheduling of jobs #1987

Merged
merged 3 commits into from Aug 25, 2016

Projects

None yet

3 participants

@jamieps
Contributor
jamieps commented Aug 5, 2016

Fixes #150.

I've used the bootstrap-datetimepicker for this, as it blended in well as Rundeck already uses Bootstrap. Since then I've noticed #1634 which uses the jQuery date picker. Do you have a preference on which one is used?

Some screenshots:

run_job_later_btn
schedule_popover
schedule_date
schedule_time

Scheduled jobs in the activity view:

activity_view

If the job supports multiple executions, then it's possible to schedule the same job multiple times. Otherwise only one execution (deferred or run immediately) is allowed.

  • Update Moment.js to 2.14.1, the later version is required for bootstrap-datetimepicker.
  • Added support for specifying the time to execute to the /api/18/job/[ID]/run endpoint. This must be a valid ISO 8601 date and time.

Some limitations...

  • Permissions are only checked when the job is scheduled, not when it is executed.
  • If Rundeck is restarted, all executions which were scheduled ad hoc will be killed and must be re-scheduled manually. This is the same behaviour as for executions which are run immediately.

We don't have access to a working cluster environment right now, though I am working towards getting one setup. Do you think takeover should affect jobs which are scheduled ad hoc?

@gschueler
Contributor

This looks great, thank you!

If Rundeck is restarted, all executions which were scheduled ad hoc will be killed and must be re-scheduled manually. This is the same behaviour as for executions which are run immediately.

I suppose if the scheduled start time is still in the future, it could be re-scheduled? The big issue with that is secure option input values, which are only passed into the scheduler context and are not stored. Perhaps if the job has no secure input options it could be rescheduled, otherwise killed. For now killing is fine.

We don't have access to a working cluster environment right now, though I am working towards getting one setup. Do you think takeover should affect jobs which are scheduled ad hoc?

Good question. Right now if you takeover a job's scheduling on another cluster node, the original node will still execute the schedule, but as soon as it is triggered it will see schedule owner has changed and unschedule it locally and then not execute. Ideally "runAt/ad-hoc" job executions would behave the same way. To implement that the "takeover schedule" action will also have to look for the one-off scheduled executions to add to quartz. Alternatively, they could be ignored and be left to the original schedule owner. Thoughts @ahonor ?

@ahonor
Contributor
ahonor commented Aug 8, 2016

Killing is fine but we should log which jobs it occurs for. It's confusing when a user expects a job to run and then gets no log of it when it doesn't execute.

As for takeover, I prefer it to act like other scheduled jobs for now.

@jamieps
Contributor
jamieps commented Aug 16, 2016 edited

@gschueler @ahonor

Killing is fine but we should log which jobs it occurs for. It's confusing when a user expects a job to run and then gets no log of it when it doesn't execute.

I've implemented rescheduling of ad hoc scheduled executions unless the job has secure input options; if a job does has secure input options, or it's not possible to reschedule the execution, an error is logged for each ad hoc scheduled execution and the execution is terminated.

As for takeover, I prefer it to act like other scheduled jobs for now.

Unscheduling and takeover now also affects ad hoc scheduled executions.

I've had to migrate some unit tests to integration tests since the claim executions functions now use criteria and I'm not aware of a way of unit testing these.

I started with using GORM named queries but then I've since read that the Grails developers have 'deprecated' this in favour of where queries instead. Do you have any preference?

I'll sort out the conflicts 'vs' master and then update the PR shortly.

@gschueler
Contributor

@jamieps awesome work, thanks!

jamieps added some commits Aug 5, 2016
@jamieps jamieps Support ad hoc scheduling of jobs
This differs from fixed scheduling because only run permissions are
necessary to schedule a job and the job only runs once at the given
time.

 - Use bootstrap-datetimepicker (for Bootstrap 3) to provide the UI
   widget to select the date/time.
 - Update Moment.js to 2.14.1, the later version is required for
   bootstrap-datetimepicker.
 - Add support for specifying the time to execute to the
   /api/18/job/[ID]/run endpoint. This must be a valid ISO 8601 date
   and time.
 - To allow multiple executions to be scheduled, the job must support
   multiple executions, otherwise only one execution (either scheduled
   or run immediately) can be spawned.

Limitations

 - Permissions are only checked when the job is scheduled, not when it
   is executed. If the run permission is revoked between the job being
   scheduled and the start time, the execution will still run (though
   it can be killed before it starts).
 - If Rundeck is restarted, all executions which were scheduled ad hoc
   will be killed and must be re-scheduled manually.
6b6c9ba
@jamieps jamieps Reschedule ad hoc scheduled executions on startup
Support cluster takeover and unscheduling of ad hoc scheduled executions
when the execution mode is switched to passive.

Some unit tests have been migrated to integration tests as functions
they call now use GORM criteria queries which cannot be unit tested.
331eeeb
@jamieps
Contributor
jamieps commented Aug 16, 2016

@gschueler I've rebased against master and updated the PR

@jamieps jamieps Ensure executions that cannot be rescheduled are killed
It is possible for cleanupRunningJobs() to complete before executions are
rescheduled, as it runs on a timer. If it wasn't possible to reschedule
an execution this could lead to orphaned executions remaining.
055920d
@jamieps
Contributor
jamieps commented Aug 17, 2016

I noticed an issue with killing off executions which can't be rescheduled.

I was relying on cleanupRunningJobs() being run at start up, however I've noticed that sometimes it completes before anything has been (re)scheduled, and in this case the executions are not killed properly.

Now cleanupRunningJobs() gets called explicitly to make sure this always happens.

@gschueler gschueler added this to the 2.6.x milestone Aug 25, 2016
@gschueler gschueler merged commit ef33994 into rundeck:master Aug 25, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@jamieps jamieps deleted the jamieps:feature/ad-hoc-scheduling branch Aug 25, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment