Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow queued jobs to be processed by a cron job #7105

Closed
NateWr opened this issue Jun 7, 2021 · 16 comments
Closed

Allow queued jobs to be processed by a cron job #7105

NateWr opened this issue Jun 7, 2021 · 16 comments
Assignees
Labels
Housekeeping:1:Todo Any dependency management or refactor that would be nice to have some day.
Milestone

Comments

@NateWr
Copy link
Member

NateWr commented Jun 7, 2021

Describe the problem you would like to solve
The job queue introduced in #4622 will be processed automatically in server environments where it is not possible to run cron jobs or service workers. However, some admins will want to process the queue through a cron job.

Describe the solution you'd like
Add support for a single cron job that will process scheduled tasks, as the current cron configuration does, as well as process jobs in the new job queue. Each server should only have to set up one cron job per instance (so one cron job can serve many journals in one installation).

The cron job to process the queue will need to execute more quickly than previous cron jobs, which were typically scheduled ro run hourly or daily. Because this jobs queue will include time-sensitive jobs, the cron configuration should be updated to fire every minute or every thirty seconds.

When migrating from 3.3 to 3.4, existing cron configurations should be considered. We may need to require admins to explicitly opt into the new cron system by modifying their config file, so that we know they have adjusted their cron job to fire at the appropriate intervals.

Who is asking for this feature?
Administrators who want to process the jobs queue from a cron job.

@NateWr NateWr added the Housekeeping:1:Todo Any dependency management or refactor that would be nice to have some day. label Jun 7, 2021
@NateWr NateWr added this to the OJS/OMP/OPS 3.4 milestone Jun 7, 2021
@henriqueramos henriqueramos self-assigned this Jun 7, 2021
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Jun 7, 2021
henriqueramos added a commit to henriqueramos/ojs that referenced this issue Jun 7, 2021
@henriqueramos
Copy link
Contributor

I've create a pull request for OJS and pkp-lib, once they're approved, I will do it for OMP and OPS.

henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 1, 2022
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 1, 2022
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 1, 2022
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 1, 2022
henriqueramos added a commit to henriqueramos/ojs that referenced this issue Feb 1, 2022
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 17, 2022
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 17, 2022
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 17, 2022
…b without asking for disable_jobs_run_at_shutdown configuration
henriqueramos added a commit to henriqueramos/ojs that referenced this issue Feb 17, 2022
henriqueramos added a commit to henriqueramos/ojs that referenced this issue Feb 17, 2022
@henriqueramos
Copy link
Contributor

@asmecher Could you please check those pull requests? If they look good enough, I will create the respect ones for OMP/OPS. Thank you!

#7106
OJS

henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 17, 2022
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 17, 2022
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 17, 2022
…b without asking for disable_jobs_run_at_shutdown configuration
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 17, 2022
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 17, 2022
henriqueramos added a commit to henriqueramos/pkp-lib that referenced this issue Feb 17, 2022
…b without asking for disable_jobs_run_at_shutdown configuration
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Aug 4, 2022
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Aug 4, 2022
touhidurabir added a commit to touhidurabir/ojs that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/omp that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/omp that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/omp that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/ops that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/ops that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/ops that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/ops that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/ops that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/pkp-lib that referenced this issue Sep 20, 2022
touhidurabir added a commit to touhidurabir/omp that referenced this issue Sep 20, 2022
@touhidurabir
Copy link
Member

@asmecher all test has passed. PRs

pkp-lib --> #8133
ojs --> pkp/ojs#3476
omp --> pkp/omp#1199
ops --> pkp/ops#348

as per per our last discussion, I have brought back the shutdown function that run one job per web request, change at https://github.com/pkp/pkp-lib/pull/8133/files#diff-d8bccac695c8cacb01f4be531434b57599b8747080dd26b56e8f1cb6d5568274R144-R151 .

asmecher pushed a commit that referenced this issue Sep 23, 2022
asmecher pushed a commit to pkp/ojs that referenced this issue Sep 23, 2022
asmecher pushed a commit to pkp/omp that referenced this issue Sep 23, 2022
asmecher pushed a commit to pkp/ops that referenced this issue Sep 23, 2022
asmecher pushed a commit to pkp/ops that referenced this issue Sep 23, 2022
asmecher pushed a commit to pkp/ops that referenced this issue Sep 23, 2022
@asmecher
Copy link
Member

Congratulations, @touhidurabir, I've merged this one!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Housekeeping:1:Todo Any dependency management or refactor that would be nice to have some day.
Projects
Status: Done
Development

No branches or pull requests

5 participants