-
Notifications
You must be signed in to change notification settings - Fork 442
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve job runner configuration/execution #9823
Comments
@jonasraoni, what documentation you mean, this one https://docs.pkp.sfu.ca/admin-guide/en/deploy-jobs? |
There was this problem before, but I've already fixed 😁 I've mixed the CLI tool with the scheduled task execution, so the documentation is ok, I've updated the description :) |
So just to be sure I understand: |
At this moment: |
Hmm... |
@jonasraoni I also think we need to have another config option to handle it . However along with this I also like to have some sort of limiter to introduce when jobs are running via cron and that can also be configured (perhaps from the Right now if a job runs via cron, it will run or try to run all the pending jobs in the list at a time . This is not bad when there is only few jobs but not an ideal situation when there is a lot of pending jobs (which has been accumulated for quite some time for some unforeseen reason) or few jobs with long run time . This may cause memory run out issue and also not very efficient . Solution 1 : we can :-
Only issue using the existing Solution 2: Another way we can go about this is to use the job runner mechanism for the cron also . and it's not much different as both use pretty much same approach but job runner have some additional constrains (such as max jobs, time, memory and possible estimation of next job to run ), underneath after applying the constrains, it runs jobs one by one . |
@asmecher @jonasraoni I like to propose the following changes we do need a new config setting for
One important decision to make , should we prioritize it for |
^ @touhidurabir and I discussed this and I'd prefer leaving this kind of change for 3.5.0. We're not making heavy use of jobs/tasks that have much overhead or turnaround time, and aren't worried about concurrent e.g. execution of the same task, as we have decent enough locking. We should be fine for 3.4.0 as-is IMO. |
@jonasraoni can you review the linked PRs at #9823 (comment) ? However this need to wait before we finalise and merge the #9678 . |
FYI I'll wait the other issue to get merged before reviewing this one. |
@jonasraoni I have updated and rebased the PR. As the #9678 got merged, it's ready for review . |
I'll take a look now, I just read the comments, and I think it's aligned with what I wrote in the description (to have a more granular control). |
#9823 update the crontrol job processing via scheduler
pkp/pkp-lib#9823 update the crontrol job processing via scheduler
pkp/pkp-lib#9823 update the crontrol job processing via scheduler
pkp/pkp-lib#9823 update the crontrol job processing via scheduler
All merged, PR and implementation details #9823 (comment) |
Describe the bug
The documentation states that jobs can be executed by:
But in fact, once the
job_runner
is enabled, both the scheduled task and the request based execution will execute the jobs.Solution
Adding a setting to control whether jobs should be executed by a scheduled task/request should be enough.
What application are you using?
OJS 3.4
PRs (main branch)
pkp-lib --> #10196
ojs --> pkp/ojs#4364
omp --> pkp/omp#1662
ops --> pkp/ops#743
Additional Information
This need to be merged to main (upcoming 3.5) after the merge of #9678
For implementation details see release note and #9823 (comment)
The text was updated successfully, but these errors were encountered: