[4.1] [SOC 21] Add a Scheduled Tasks Infrastructure to Joomla

[ditsuke]

Project Repository: joomla-projects/soc21_website-cronjob.

Introduction

Many web hosting services do not offer access to the native UNIX cron. A popular solution to scheduling tasks with such servers is using a lazy scheduler (or, 'pseudo-cron'). Other major CMS like Drupal and WordPress already offer some form of pseudo-cron functionality.

However, there is currently no standard way to schedule tasks in the Joomla! core. There is also no way to integrate native cron with Joomla! if the server offers access to this functionality.

This PR introduces a Scheduled Tasks infrastructure to Joomla! As of the last update to this description, this infrastructure includes a Scheduler component, a system plugin to run the Scheduler on each page load implemented in the first half of my work under SOC '21. Also included is a test task plugin along with 2 functional job plugins:

• Plugin Task - Requests offers a GET Request task routine with the ability to set authorization headers and a timeout.
• Plugin Task - Site Status offers task routines to change the site's offline status. This can be very useful with say a Cron-style execution rule, so it can be used to turn a site offline for a specific timeframe.

Summary of Changes

1. Scheduler Component: The component provides the infrastructure for Scheduled Tasks, which includes the MVC to manage tasks, the plugin API necessary for plugins to advertise and run task routines, a helper for execution rules (currently supported: intervals and cron-style expressions), a trait for task plugins, and a Scheduler API class.
2. Cronjobs System Plugin: Subscribes to the onBeforeRender event, and registers a method to run the Scheduler as a shutdown_function. At most a single scheduled task is executed per request. The task execution is logged to the joomla_scheduler.php log file.
3. Requests Task Plugin: Offers a task routine for GET Requests with support for auth headers and request timeout.
4. Site Status Task Plugin: Offers task routines to change the site's offline status.
5. Database:

• Adds a #__scheduler_tasks database table.
• Adds com_scheduler as an #__assets entry.
• Adds #__extensions entries for com_scheduler, plg_system_schedulerunner, plg_task_demotasks, plg_task_requests, plg_job_sitestatus.

6. System Dashboard: Adds the Scheduled Tasks Manager under the Manage module.

Planned Work

Right now, the major planned milestones include:

• Joomla Console commands:
  • scheduler:run
  • scheduler:list
  • scheduler:add
  • scheduler:state
• Email notifications.
• Joomla API MVC,
• Integration with native Cron if available,
• Plugin for script or arbitrary command execution,

Enhancements and Fixes

• Rebranding to Scheduler. Ref [1] [2]
• Table indexes Ref / @PhilETaylor
• Cronjob priority Ref / @PhilETaylor
• User action-log triggers for management. Ref / @PhilETaylor
• Hide trashed jobs from default filtered state. Ref / @scout507
• Events for job success/failure. Ref / @PhilETaylor
• Optional per-job log file. Ref / @PhilETaylor

Testing Instructions

Management Workflow

For testing, a fresh installation/config is important. Steps:

1. Go to System Dashboard > Manage module > Scheduled Tasks

3. Click on the button to add a new Task.

4. Select a task routine as the basis of your task.

5. Configure execution rules. We have two distinct options: interval based rules and a "custom" cron-style rule.

6. And some more (advanced) configuration:

6. Save the task. Load a page on the site when you expect the task to execute.

7. The job should execute and it should appear in the joomla_scheduler.php log file.

The log should look like this:

#Fields: date	time	priority	message
2021-10-31	23:03:30	INFO	Running task#03 'Test Task'.
2021-10-31	23:03:31	INFO	Task> Request response code was: 200
2021-10-31	23:03:31	INFO	Successfully finished task#03 in 0.90 (net 0.91) seconds.

@todo: testing instructions for the CLI and Webcron.

Documentation Changes Required

We'll need all new documentation for the Scheduler, the Tasks infrastructure

Acknowledgements

I would like to give a special thanks to @alikon for their work on #29592. I have and continue to draw much inspiration from their ideas, many of which I've indirectly or directly used from the very nascent stages of development to now especially with the system plugin and the console commands in development.

I also owe much gratitude to my mentor @bembelimen who is always there to help me with things even at the oddest of times. A special mention also to our team lead @shivamdiehard who makes sure things go over smoothly.

[brianteeman]

Please review and correct the codestyle used in all the xml

[brianteeman]

Looking at the language file I can see a lot of values which could be replaced with existing strings from global language files. This will reduce the workload on translators and also ensure a more consistent ui.

For example COM_CRONJOBS_FIELDSET_RULES is the string used to display a fieldset called JCONFIG_PERMISSIONS_LABEL

[scout507]

The feature seems to work just fine and is easy to use. The only issue I encountered has to do with the input fields on the 'new crownjob' page:

• negative input in the interval field leads to an error
• if you fill in all the necessary fields, but one with an illegal input, the 'Job Parameters' fields go missing:
  

Also, if you trash an existing crownjob, it will still apear in the crownjob list. Not sure if this is intended, but it's not realy clear where the diffrence between trashing and disabling is.

[socke300]

First of all

Hey, the cronjobs work well and the UI is simple and intuitive to use.

That's how I tested it

I went through the different cronjobs, created some and checked afterwards if they work. I also tested different inputs for the input options, for example the intervals.

This is what I found

Like the previous speaker, I noticed the inputs. There is also an error here if the inputs are too long for the intervals or if they contain a point/comma. In general, I think the times should not be written with a comma, there should be a separate field for seconds/minutes/hours or you can specify a format, for example day:hour:minute:second. Otherwise you could not, for example, enter 1:30 hours/minutes as a time period. I also noticed an unimportant detail in the UI, when you create a new cronejob, the plus becomes white, this should perhaps be blue or something. (But that's not important)

This happened when the input was too long.

This happened when entering a certain length.

Result

All in all, the feature is very nice and it works very well despite the little things. (They were really only small things)

[brianteeman]

Please ask your mentors how to format and indent the xml files.

[alikon]

changing the state from CLI doesn't work

[alikon]

running the test task from admin

Last run not updated it is expected ?

[bembelimen]

Thank you @ditsuke , @brianteeman , @PhilETaylor , @richard67 , @HLeithner , @Denitz , @alikon , @dgrammatiko , @nibra , @chmst , @softforge (and many more) who made this possible and happen.

[alikon]

from when a pr it's merged without 2 test
...really surprised..