- The buildout infrastructure
- Note for tests
- collective.cron 1.0 => collective.cron 2.0
- Detailled documentation
collective.cron is a cron-like asynchronous tasks system based on top of plone.app.async and plone.app.registry. The implementation hasn't not for now all the bells and wistles of a nice UI. However the simple interface do all the stuff And The underlying job manager works reliably. Finaly, you can register your tasks easily.
Note that at the moment, we have 100% test coverage. This do not prevent from bugs but that make us comfurtable from the major hazard we can have.
The design is modern and modular, imagine that you can even easily change from plone.app.async to another job system.
- base.cfg -> base buildout informations
- buildout.cfg -> base buildout for the current plone version
- test-4.0.x.cfg -> test buildout for plone4.0
- test-4.1.x.cfg -> test buildout for plone4.1
- test-4.2.x.cfg -> test buildout for plone4.2
The most important things are in base.cfg. If you plan to integrate plone.app.cron to your buildout, please refer to the plone.app.async documentation.
- For now we use the unreleased version of plone.app.async : https://github.com/plone/plone.app.async
Tests can unpredictibly crash because of monkey patchs to datetime. This is a false positive. Just relaunch them if you see something similar
ConflictError: database conflict error (oid 0x2549d9dd3cf6b59b, serial this txn started with 0x0399e4b3adb993bb 2012-10-14 09:23:40.716776, serial currently committed 0x0399e4b3ae733c77 2012-10-14 09:23:40.886752)
in 1.0, each cron task was a content. This was then tedious to replicate and maintain accross multiple instances and plone versions. One of the goal of collective.cron 2.0 is to avoid at most to have persistance, specially specialized contents to minimize all the common migration problems we have with an objects database. Thus a choice has been made to heavily use plone.app.registry as a configuration backend.
Thus, there is no migration prepared from collective.cron 1.0 to 2.0 It is up to you to do it. Specially, you will have to clean the database of all specific collective.cron 1.0 based & persistent content before upgrading. Indeed, As the design of tasks is really different, we can't do any automatic migration.
First with collective.cron 1 in your buildout
- Search, record settings then delete all IBackend content
- Delete all jobresults & persistent contents
- Cleanup all the zc.async queue
In a second time, deactivate collective.cron1 and activate collective.cron 2+ in your buildout
- Adapt your adapters and content types to work with collective.cron 2.0 (inputs/mark items to work on)
- add equivalent crons records to the crontab setting of the backends job
- kiorky <email@example.com>
- collective.cron lets you register crons which run periodically in your system.
- Each plone site has a crontab.
- This crontab is used by many composants to execute the cron jobs.
- We have a central dashboard which will list all tasks registered on the site crontab.
- The tasks configuration is based on plone.app.registry but design to be replaceable (composant)
- The tasks execution is based on plone.app.async but design to be also replaceable (composant)
- The cron manager will ensure to restore all cron jobs for all plone sites at zope restart.
A crontab is the collection of all cron registered to a plone site. A crontab can be (de)activated globally Each crontab sub element (the crontab, the crons & associated logs) defines a dump method which make a JSON representation of the object.
The major attributes for a crontab are:
- crons: An ordered dict of crons. Key is the cron uid
- activated: globally power switch for the crontab
- manager: the manager is responsible for the crontab persistence
- save(): save the crontab
- save_cron(cron): save the cron
When a crontab is saved, it emits a
The major attributes for a cron are:
- name: will be the queried name to search jobs
- periodicity: give the next time execution
- environ: An optionnal jsonencoded mapping of values which will be given to the task
- logs_limit: logs to keep (default : 5, limit : 25)
- uid: internal id for the crontab machinery
- user: The user the task will run as, its up to you to make the task run as this user
- activated: the activation status of the cron
- logs: give the last logs of the cron prior executions from most recent to older.
- crontab: A possibly null reference to the parent crontab
A note on the user which is only a stocked value. you can see
collective.cron.utils.su_plone to help you switch to that user.
IT IS UP TO YOU TO SWITCH TO THAT USER IN YOUR JOBRUNNER.
The major attributes for a log are:
- date: date of logging
- status: status ::= NOTRUN | FAILURE | WARN | OK
- message: the logs
Based on top of plone.app.registry, collective.cron record the crontab current status in the site registry. It adapts a crontab.
- activated: Boolean switch status of the crontab
- cronsettings: the raw manager settings (.crontab, .activated)
- crons: list of serialized strings representations of the crons
- read_only: if true, changes will be a NOOP
When a record is touched (added, edited, removed), we fire an event to syncronize the queue.
This composant is responsible when a CrontabSynchronisationEvent is fired to synchronise the crontab with the job queuing system. It will remove unrelated jobs and schedule new jobs. It adapts a plonesite and a crontab.
When the crontab save itself, its emits a
ModifiedCrontabEvent which in turns is redirected as a
CrontabSynchronisationEvent to let the manager synchronize the queue.
When the server restarts, a
ServerRestartEvent is called to re-register any cron job that would have been wiped from the queue.
This composant is responsible forthe execution and presence in the queue of a particular cronjob.It can register or remove the job execution of a cron. This is a firendly proxy to the 'Queue manager"
It adapts a plonesite and a cron.
When it register a cronjob, the job queued is a cron jobrunner wrapper responsible for:
- Sending a
- Running the relevant JobRunner (a named adapter adapting the plonesite, and the cron)
- Sending a
- logging the execution
- Scheduling the next execution
- A cron jobrunner is a named adapter which:
- adapts the plonesite and the current cron
- implements IJobRunbner, and specially defines a run method.
A base class exists in collective cron, just inherit from it This is a complicated definition to have a class like this:
from collective.cron import crontab class MyCronJob(crontab.Runner): def run(self): print "foo"
registered in zcml like that:
<adapter factory=".module.MyCronJob" name="mycronjob"/>
Annd then, you ll have to register cron called
mycronjob in your plonesite,
This composant will manage the jobs inside the job queue. You ll have enought methods to know for a specific cron if a job is present, what is its status, You can also register, or delete items from the running queue It adapts a plonesite.
Responsible to mark infos in the async queue to make posible the reload of jobs at Zope restart.
There are 3 ways to register tasks:
- via the API
- via the UI
- via Generic Setup (profile)