This package contains the Job-Runner Worker, which is responsible for executing the scheduled jobs managed by the Job-Runner.
Requirements (depending on your distro, the naming might be a bit different):
Then install this package by executing
python setup.py install or
python setup.py develop (for development). In the latter, you might want
to install the testing requirements by executing
pip install -r test-requirements.txt.
See the getting started section in the Job-Runner documentation ( in the job-runner repo) for setting up the whole project.
Example with required settings:
[job_runner_worker] api_base_url=http://domain.of.job.runner/ api_key=worker1 secret=verysecret script_temp_path=/tmp ws_server_hostname=domain.of.websocket.server broadcaster_server_hostname=domain.of.broadcast.server
All available settings
- The base URL which will be used to access the API. This should start with
- Public-key to access the API.
- Private-key to access the API.
- The number of jobs to run concurrently. Default:
The log level. Default:
'info'. Valid options are:
- The maximum number of bytes of the log that is sent back to the API. This
is to avoid
413 Request Entity Too Largeerrors. If the log will be larger than this value, 20% of the allowed size will be taken from the top of the log, the remaining 80% will be taken from the bottom. Everything in between will be truncated. Default:
- The hostname of the WebSocket Server.
- The port of the WebSocket Server. Default:
- The path where the scripts that are being executed through the Job-Runner are temporarily stored. Note that this should be a location where executable scripts are allowed!
- The hostname of the queue broadcaster server.
- The port of the queue broadcaster server. Default:
- Seconds after which the subscriber is re-connecting to the publisher
when no data has been received. Default:
300. This is useful when you are loadbalancing the publisher and it keeps the TCP connection open on the front-end, when the connection on the back-end has been closed. Because of this ZMQ doesn't detect that it is not connected anymore and jobs get stuck.
For starting the worker, you can use the
usage: job_runner_worker [-h] [--config-path CONFIG_PATH] Job Runner worker optional arguments: -h, --help show this help message and exit --config-path CONFIG_PATH absolute path to config file (default: SETTINGS_PATH env variable)
- The worker will now terminate gracefully when receiving the
TERMsignal. This means that all pending jobs will be completed, but that it will not accept any new jobs. After finishing the last pending job, the worker will terminate.
reconnect_after_inactivitydefault to 10 minutes. This is 2 x the
JOB_RUNNER_WORKER_PING_INTERVALdefault setting in Job-Runner.
- Implement handler for
- Add and implement
- Run script by finding their shebang without the x bit being needed.
- Handle separate run log-output resource. This requires Job-Runner >= v1.3.0.
- Fix killing job-runs. Where v1.0.5 was killing children processes, it did not kill children of children, ... This should kill the full tree of child-processes.
- Freeze requests library version, since 1.0.0 contains backwards compatible changes.
- Fix killing job-runs. When the process had sub-processes, only the parent process was killed and the worker was waiting for the child-processes to complete.
- Add config variable
max_log_bytesto limit the amount of logdata that will be send back to the API (to avoid
413 Request Entity Too Largeerrors).
pidback to the REST API when a job has been started.
- Kill a job-run when a
killaction is received.
- Make sure that the API exactly matches.
- Make the timezones send to the REST API timezone aware.
- Deployar related changes.
- Fix encoding issue when writing the file.
- Refactor to make the worker compatible with the 0.7 version of the
- Make it consume runs from the queue broadcaster instead of hitting the REST interface every x seconds.
- Add retry on error to recover from temporary REST interface errors.
- Merge fixes v0.5.1 and v0.5.2 into v0.6.x version.
- Refactor to make use of separate WebSocket Server.
- Make temporary path for scripts configurable.
- Disable SSL certificate validation.
- Initial release.