A really lightweight Clojure scheduler
Clojure
Latest commit f4a9087 Dec 27, 2016 @e0d e0d committed with updating readme in light of deprecation

README.org

Chime

Chime is a really lightweight Clojure scheduler.

(Despite the last modification date, Chime is still being actively used and maintained - just that there isn’t a lot of maintenance to do!)

Dependency

Add the following to your project.clj file:

[jarohen/chime "0.2.0"]

The ‘Big Idea’™ behind Chime

The main goal of Chime was to create the simplest possible scheduler. Many scheduling libraries have gone before, most attempting to either mimic cron-style syntax, or creating whole DSLs of their own. This is all well and good, until your scheduling needs cannot be (easily) expressed using these syntaxes.

When returning to the grass roots of a what a scheduler actually is, we realised that a scheduler is really just a promise to execute a function at a (possibly infinite) sequence of times. So, that is exactly what Chime is - and no more!

Chime doesn’t really mind how you generate this sequence of times - in the spirit of composability you are free to choose whatever method you like! (yes, even including other cron-style/scheduling DSLs!)

When using Chime in other projects, I have settled on a couple of patterns (mainly involving the rather excellent time functions provided by clj-time - more on this below.)

Usage

Chime consists of two main function, chime-ch and chime-at.

Here we are making use of clj-time’s time functions to generate the sequence of Joda times.

chime-ch

chime-ch is called with an ordered sequence of Joda times, and returns a channel that sends an event at each time in the sequence.

(:require [chime :refer [chime-ch]]
          [clj-time.core :as t]
          [clojure.core.async :as a :refer [<! go-loop]])

(let [chimes (chime-ch [(-> 2 t/seconds t/from-now)
                        (-> 3 t/seconds t/from-now)])]
  (a/<!! (go-loop []
           (when-let [msg (<! chimes)]
             (prn "Chiming at:" msg)
             (recur)))))

chime-ch uses an unbuffered channel, so cancelling a schedule is achieved simply by not reading from the channel.

You can also pass chime-ch a buffered channel as an optional argument. This is particularly useful if you need to specify the behaviour of the scheduler if one job overruns.

core.async has three main types of buffers: sliding, dropping and fixed. In these examples, imagining an hourly schedule, let’s say the 3pm run finishes at 5:10pm.

  • With a sliding-buffer (example below), the 4pm job would be cancelled, and the 5pm job started at 5:10.
  • With a dropping-buffer, the 4pm job would start at 5:10, but the 5pm job would be cancelled.
  • In the unbuffered example, above, the 4pm job would have been started at 5:10pm, and the 5pm job starting whenever that finished.
(:require [chime :refer [chime-ch]]
          [clj-time.core :as t]
          [clojure.core.async :as a :refer [<! go-loop]])

(let [chimes (chime-ch times {:ch (a/chan (a/sliding-buffer 1))})]
  (go-loop []
    (when-let [time (<! chimes)]
      ;; ...
      (recur))))

You can close! the channel returned by chime-ch to cancel the schedule.

chime-at

chime-at, on the other hand, is called with the sequence of times, and a callback function:

(:require [chime :refer [chime-at]]
          [clj-time.core :as t])

(chime-at [(-> 2 t/seconds t/from-now)
           (-> 4 t/seconds t/from-now)]
          (fn [time]
            (println "Chiming at" time)))

With chime-at, it is the caller’s responsibility to handle over-running jobs. chime-at will never execute jobs of the same scheduler in parallel or drop jobs. If a schedule is cancelled before a job is started, the job will not run.

chime-at returns a zero-arg function that can be called to cancel the schedule.

You can also pass an on-finished parameter to chime-at to run a callback when the schedule has finished (if it’s a finite schedule, of course!):

(chime-at [(-> 2 t/seconds t/from-now) (-> 4 t/seconds t/from-now)]

          (fn [time]
            (println "Chiming at" time))

          {:on-finished (fn []
                          (println "Schedule finished."))})

Recurring schedules

To achieve recurring schedules, we can lazily generate an infinite sequence of times using the new (as of 0.5.0) clj-time periodic-seq function. This example runs every 5 minutes from now:

(:require [clj-time.core :as t]
          [clj-time.periodic :refer [periodic-seq]])

(rest    ; excludes *right now*
 (periodic-seq (t/now)
               (-> 5 t/minutes)))

To start a recurring schedule at a particular time, you can combine this example with some standard Clojure functions. Let’s say you want to run a function at 8pm New York time every day. To generate the sequence of times, you’ll need to seed the call to periodic-seq with the next time you want the function to run:

(:require [clj-time.core :as t])
(:import [org.joda.time DateTimeZone])

(->> (periodic-seq (.. (t/now)
                       (withZone (DateTimeZone/forID "America/New_York"))
                       (withTime 20 0 0 0))
                   (-> 1 t/days)))

Chime does drop any times that have already passed from the front of your sequence of times (on the condition that the sequence is ordered) so it doesn’t matter whether 8pm today has already passed - Chime will handle this gracefully.

Complex schedules

Because there is no scheduling DSL included with Chime, the sorts of schedules that you can achieve are not limited to the scope of the DSL.

Instead, complex schedules can be expressed with liberal use of standard Clojure sequence-manipulation functions:

(:require [clj-time.core :as t])
(:import [org.joda.time DateTimeConstants DateTimeZone])

;; Every Tuesday and Friday:
(->> (periodic-seq (.. (t/now)
                       (withZone (DateTimeZone/forID "America/New_York"))
                       (withTime 0 0 0 0))
                   (-> 1 t/days))
     (filter (comp #{DateTimeConstants/TUESDAY
                     DateTimeConstants/FRIDAY}
                   #(.getDayOfWeek %))))

;; Week-days
(->> (periodic-seq ...)
     (remove (comp #{DateTimeConstants/SATURDAY
                     DateTimeConstants/SUNDAY}
                   #(.getDayOfWeek %))))

;; Last Monday of the month:
(->> (periodic-seq (.. (t/now)
                       (withZone (DateTimeZone/forID "America/New_York"))
                       (withTime 0 0 0 0))
                   (-> 1 t/days))

     ;; Get all the Mondays
     (filter (comp #{DateTimeConstants/MONDAY}
                   #(.getDayOfWeek %)))

     ;; Split into months
     ;; (Make sure you use partition-by, not group-by -
     ;;  it's an infinite series!)
     (partition-by #(.getMonthOfYear %))

     ;; Only keep the last one in each month
     (map last))

;; 'Triple witching days':
;; (The third Fridays in March, June, September and December)
;; (see http://en.wikipedia.org/wiki/Triple_witching_day)

;; Here we have to revert the start day to the first day of the month
;; so that when we split by month, we know which Friday is the third
;; Friday. (Any times that have already passed will be dropped, as
;; before)

(->> (periodic-seq (.. (t/now)
                       (withZone (DateTimeZone/forID "America/New_York"))
                       (withTime 0 0 0 0)
                       (withDayOfMonth 1)
                       (-> 1 t/days))

                   (filter (comp #{DateTimeConstants/FRIDAY}
                                 #(.getDayOfWeek %)))

                   (filter (comp #{3 6 9 12}
                                 #(.getMonthOfYear %)))

                   ;; Split into months
                   (partition-by #(.getMonthOfYear %))

                   ;; Only keep the third one in each month
                   (map #(nth % 2))))

This is quite a different approach to other scheduling libraries, and therefore I would be very interested to hear your thoughts!

Error handling

As of 0.1.1, you can pass an error-handler to chime-at - a function that takes the exception as an argument. You can either re-throw it, to prevent future occurrences of the scheduled task; or squash it to try again at the next scheduled time.

By default, Chime will re-throw the error to the thread’s uncaught exception handler.

(chime-at [times...]
          do-task-fn
          {:error-handler (fn [e]
                            ;; log, alert, notify etc?
                            )})

Behaviour of (t/now)

Sometimes, you’ll want a schedule along the lines of ‘every <x> <time-unit>’. The temptation here is to create a sequence of times with: (periodic-seq (t/now) (t/minutes 5)) - however, this can lead to non-deterministic behaviour. Sometimes Chime will run the function immediately, sometimes it won’t.

The reason for this is a combination of two factors:

  • Chime removes times in the past from your sequence. This is so that, when you want a schedule like ‘6am daily’, you can pass (periodic-seq (t/today-at 6 0) (t/days 1)), without worrying whether 6am has already passed today.
  • There’s a slight delay between your call to (t/now) and Chime’s check for times in the past. Chime (like joda-time) resolves times to the nearest millisecond so, if these two checks occur in the same millisecond, your schedule will run immediately - if not, it won’t.

The solution to this is to exclude (t/now) from the schedule - achieved with something like (rest (periodic-seq (t/now) (t/minutes 5))).

Testing your integration with Chime

Testing time-dependent applications is always more challenging than other non-time-dependent systems. Chime makes this easier by allowing you to test the sequence of times independently from the execution of the scheduled job.

(Although, don’t forget to wrap your infinite sequences with (take x ...) when debugging!)

Bugs/thoughts/ideas/suggestions/patches etc

Please feel free to submit these through Github in the usual way!

Thanks!

Contributors

A big thanks to all of Chime’s contributors, a full list of whom are detailed in the Changelog.

License

Copyright © 2013 James Henderson

Distributed under the Eclipse Public License, the same as Clojure.

Big thanks to Malcolm Sparks for providing the initial idea, as well as his other contributions and discussions.