Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OMT-215 batch platform #17

Merged
merged 3 commits into from Aug 3, 2020
Merged

OMT-215 batch platform #17

merged 3 commits into from Aug 3, 2020

Conversation

edarchis
Copy link
Member

@edarchis edarchis commented Aug 3, 2020

Depends on changes in openimis-be_py

Adds the APscheduler for batch execution. Each module can create its own tasks that can be configured from settings.py file so that each administrator can tune it but also configure it from the environment.

There are two configurations:

  • A module can expose a task and have it configured in the settings.py SCHEDULER_JOBS.
  • For more advanced scheduling, the module can expose a method that will handle the scheduling itself. It can still accept any number of parameters that will be usable in the SCHEDULER_CUSTOM of settings.py

Example of settings:

SCHEDULER_JOBS = [
    {
        "method": "core.tasks.openimis_test_batch",
        "args": ["cron"],
        "kwargs": {"id": "openimis_test_batch", "minute": 54, "replace_existing": True},
    },
]
# This one is called directly with the scheduler object as first parameter. The methods can schedule things on their own
SCHEDULER_CUSTOM = [
    {
        "method": "core.tasks.sample_method",
        "args": ["sample"],
        "kwargs": {"sample_named": "param"},
    },
]

By default, the scheduler will run within the main Django instance. To prevent that, just set SCHEDULER_AUTOSTART=False To start the scheduler in its own process, simply use the command manage.py runapscheduler

The regular way to enable the schedule in Django is to call a method from the Config.ready(). However, this is not very flexible and doesn't work with our modular architecture. So the core takes a couple of lists from the settings and dynamically invokes the corresponding methods.
There are two configuration lists: jobs to be added and customer scheduler methods if a simple add_job is not enough
The @shared_task annotation is optional.
@edarchis edarchis requested a review from xgill August 3, 2020 06:55
@lgtm-com
Copy link

lgtm-com bot commented Aug 3, 2020

This pull request introduces 2 alerts when merging 1df1abc into bc63b4f - view on LGTM.com

new alerts:

  • 1 for Non-callable called
  • 1 for Unused import

@xgill xgill merged commit 612be16 into develop Aug 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants