Scheduler Action

[motorina0]

• each time a cron job is triggered it should invoke one action
• the action is a REST HTTP call
  • please see below some screens from Postman
  • we can have something similar
  • the example below calls an API that updates a certain extension to the latest version
    • the API does not exist yet, is for demo purposes only
• we should not add support for OS wise actions (like rmdir) for now (keep it simple(r))
• the result of the job execution should be saved in a table (HTTP success status and response body if any)
  • this way the user can check that the action was executed at the correct time with the correct params

Call Config:

Call Body

[motorina0]

In the backend a call of this form will be made each time a job is executed:

async with httpx.AsyncClient() as client:
    resp = await client.get(    # action can be any HTTP verb (GET, PUT, POST, etc) set in the job config
        url,                    # from the job config
        headers,                # from the job config
    )
    save_job_execution(resp.status_code, resp.text())

[bitkarrot]

@motorina0 OK. I think the directions are clear.

The python-crontabs module would store the X-API-Key values as environment variables in the crontab file, so each individual job will have its own set of env vars.

Assuming there is no need for global env vars, I'll remove the Global Environment variables box that is currently at the bottom of the extension right now.

Please do let me know if incorrect! thanks.

[motorina0]

Regarding running jobs:

• there should be only one command file
• the job_id is passed to the command file
• the command file fetches the full job details from the DB and executes the HTTP call

Adapting a simple example from here:

from crontab import CronTab

cron = CronTab(user='username')
job = cron.new(command='python lnbits/utils/cron-job.py') # pass the job_id param somehow
job.minute.every(1)

cron.write()

• cron-job.py:

# imports go here

# extract job_id param
# query DB for job

async with httpx.AsyncClient() as client:
    resp = await client.get(    # action can be any HTTP verb (GET, PUT, POST, etc) set in the job config
        url,                    # from the job config
        headers,                # from the job config
    )
    save_job_execution(resp.status_code, resp.text())

Note:

• I'm not sure if the job creation example is the latest and greatest way of starting jobs with the crontab extension.
• its just to illustrate the flow

[bitkarrot]

Working on testing the extension install now.

Any suggestions on where to put the dependency, would be helpful.
Right now the python-crontab dependency is in the requirements.txt file but the user has to install it themselves, its not on the LNBits main poetry toml.

Also the user must set the username that has access to system /usr/bin/crontab
and they need to specify their LNBITS Install, as the crontab script needs to call the API and does not inherit the LNBits environment. I've put these two variables in a .env file, but there is no UI to edit this directly right now.

Implemented as you suggested using a run_cron_job.py script.

[motorina0]

Any suggestions on where to put the dependency, would be helpful.

• please open a PR on the lnbits/lnbits repo with the new dependency
• this PR should be merged before releasing the extension

as the crontab script needs to call the API and does not inherit the LNBits environment

• relative paths should be supported in the front-end
• if a relative path is used then the back-end must add http://${HOST}/${PORT}

Also the user must set the username that has access to system /usr/bin/crontab

• why is a OS user required? what happens on non-linux system?

[bitkarrot]

According to the docs, there are three ways to do it, with Linux.
One of the ways requires the username.

Do we want to enable the username? I know for OSX, I needed it.

[motorina0]

Do we want to enable the username?

• we want to keep this as simple as possible for the users that install the extension
• it looks like the username option is not mandatory, that is good

I know for OSX, I needed it.

• you mean you had to take extra steps in order to use the lib?
• we should limit/automate that as much as possible

[bitkarrot]

OK I am testing this with both Ubuntu and OSX. Will try to have everything working with latest 0.11 this weekend. (this week has been a bit busy)

[bitkarrot]

* if a relative path is used then the back-end must add `http://${HOST}/${PORT}`

Regarding - I have this script running when the cron job is triggered but is does not have the LNBits environment, I put it in as an env variable in .env but if you have a better way of accessing it, please suggest!

scheduler/run_cron_job.py

Line 20 in 2ceb03e

LNBITS_BASE_URL = os.environ.get('LNBITS_BASE_URL') or 'http://localhost:5000'

extension updated to allow for LNBits v0.11 but still need to do more testing.

[motorina0]

Regarding - I have this script running when the cron job is triggered but is does not have the LNBits environment, I put it in as an env variable in .env but if you have a better way of accessing it, please suggest!

• .env file has these properties:

HOST=127.0.0.1
PORT=5000

• data can also be passes to the job in the comment field (or something similar), right?

[bitkarrot]

HOST=127.0.0.1
PORT=5000

* data can also be passes to the job in the `comment` field (or something similar), right?

yes I know that the HOST/PORT is in the LNBits .env, but that cron script unfortunately runs outside of the LNBits environment as its called by system crontab which doesn't inherit the LNBits environment.

I can pass the HOST/PORT in, just like the job ID and adminKey is passed in right now (this is testnet example).

Perhaps this is better way to do it.

Ok I will test and get back to you. Thanks @motorina0

[bitkarrot]

OK I fixed the above, made a release. It should be working with basic functionality now, validation can improve but please have a look. https://github.com/bitkarrot/scheduler/releases/tag/v0.0.9, @motorina0

[bitkarrot]

tested with sqlitedb only so far, no postgres