A feature suggestion, spread out the schedule.

[ecrode]

Not really an issue, but I have many users who I recommend to use your site to automate interaction with my service. Most of them choose to run the job "every 5 minutes" because it is convenient option.

The side effect of this is that I get spikes every 5 minutes and I noticed that sometimes this results in an error "Last status: Failed (503 Service Temporarily Unavailable)", I guess I need to upgrade my server :-)

In any case, I do wish to suggest something which may prove useful to improve quality of service: When a user chooses "every X minutes", cron-job service could generate a (random number % X) and use it to spread out user requests.

So for example, lets say we have 100 users, all of who chose to run the cron-job every 5 minutes. Instead of having all of them run at 1PM, 1:05PM, 1:10PM... each one would get a random number (at the time a cron-job is created) from 0 to 4 which would be used to offset their timings so that 20% of them run at original 1:00, 1:05, etc; another 20% would run at 1:01, 1:06, etc; another 20% would be at 1:02, 1:07... etc ...

A good idea would be to give users an option to override this, so they have an option of locking the offset at 0 or 3 or ... , but that option is not enabled by default.

Edit: As an alternative, it would be nice to give a user an option to specify the timings as a string. This could be a standard cron (link) formatted string or something of your own design. This way I could analyze my server usage and provide a suggested configuration when referring users to use your service. This would be essentially another way to specify a "User Defined" timings.

Thank you very much! If there is anything I can do to help with coding and / or testing this, please let me know.
Gene

[tdmalone]

This is sort of like jitter for minute-by-minute schedules... but by minute instead.

Anyway, perhaps it might be possible to suggest to users to choose a random minute themselves? Something like */6 would be every 6 minutes (or thereabouts). Or they could enter 4,9,13,18 etc.? I suggest this because a random minute isn't really part of the way cron usually works anyway.

[pschlan]

Hi,

thanks - sounds like a very reasonable idea.

From the implementation point of view it's not trivial to do this, at least not when the random spreading value should vary on every request. I'll need to think about how we can achieve this while still having good performance in the job selection routine.

The alternative you suggested in your edit is already covered by issue #12, if I understand it correctly.

I'll update this issue once I have an idea how to implement this efficiently.

Patrick

[ecrode]

Hi Patrick, yes the alternative I've mentioned sounds similar to #12.

I don't think that random spreading has to be applied to every request, just when the job is setup.
Basically, during the setup, and in case every X minutes is chosen, convert it to a custom list of minutes, virtually identical to the user using a "user defined" option. Choose an offset = rand % X and set minutes to offset + (X * n) where n is = 0 to ((60 / X)-1).

So if the user has chosen ever 5 minutes, and offset = rand % 5 = 3, than cron-job would run every hour at 3, 8, 13, 18... without needing to redo the calculation / setup.
Another user with 5 minute job may get a 1 as an offset, resulting in their job running at 1, 6, 11, 16...

For advanced users, you may need to allow an option to force the offset to 0, but for every day users offset will not matter in the slightest.

Thanks again,
Gene

[ecrode]

Another food for thought, is that offset could be supplied by the target server.

If the user wants to setup cron-job with url: dom.com/some/page.php to run every 10 minutes
Than cron-job.org could send a request to dom.com/cron-api/new.php?run=10 and receive an offset within response body, such as "+4"

Alternately a custom header could be send by the server familiar with cron-job.org, such as Cron-offset: +2 to change schedule for all future requests by +2 minutes, ideally negative numbers would also be allowed (such as -3) and in the perfect world offset could be specified in milliseconds.

The changes would be made to the schedule, not to individual requests. I think it would be wrong to think about it as "request = schedule + offset", instead it should work like "schedule = schedule + offset" and continue running "request = schedule"

Thanks,
Gene

[ecrode]

Sorry to keep adding things to consider, but I'm just listing all of the options, maybe one will be easier than the other.

For jobs which fail due to error 503, it may be a good idea to automatically retry them 30 seconds later, during off-peak so to speak.

While on this subject, it may be a good idea to avoid sending a notification for the first 503 error, most of the time it is temporary and user can do little about it. Two in a row on the other hand, notify away.

Thanks again,
Gene