Skip to content

Latest commit

 

History

History
273 lines (187 loc) · 5.66 KB

README.md

File metadata and controls

273 lines (187 loc) · 5.66 KB
Raw

Github Watcher Backend

Asynchronous application for checking triggers and performing reactions.

Usage

Settings

Can be set by both yaml file and environment variables, example. File path can be set by GITHUB_WATCHER_SETTINGS_YAML environment variable. Settings consist of next sections:

  • app - general application settings
  • logs - logging settings
  • tasks - task processing settings

App

app.env - application environment, used mainly for logging. Default is production.

app:
  env: development

Can be set by GITHUB_WATCHER_APP__ENV environment variable.

app.debug - debug mode, enables more verbose logging. Default is false.

app:
  debug: true

Can be set by GITHUB_WATCHER_APP__DEBUG environment variable.

Logs

logs.level logging level, can be one of debug, info, warning, error. Default is info.

logs:
  level: debug

Can be set by GITHUB_WATCHER_LOGS__LEVEL environment variable.

logs.format - logging format string, passed to python logging formatter. Default is %(asctime)s - %(name)s - %(levelname)s - %(message)s.

logs:
  format: "%(message)s"

Can be set by GITHUB_WATCHER_LOGS__FORMAT environment variable.

Tasks

tasks.config_backend - backend configuration, used for setting up triggers and reactions. Can select among different types. Currently, only yaml_file is supported.

tasks:
  config_backend:
    type: yaml_file
    path: example/config.yaml

Can be set by environment variables:

GITHUB_WATCHER_TASKS__CONFIG_BACKEND__TYPE=yaml_file
GITHUB_WATCHER_TASKS__CONFIG_BACKEND__PATH=example/config.yaml

tasks.queue_backend - queue configuration, used as a message broker for task processing. Can select among different types. Currently, only memory is supported.

tasks:
  queue_backend:
    type: memory

Can be set by environment variables:

GITHUB_WATCHER_TASKS__QUEUE_BACKEND__TYPE=memory

tasks.state_backend - state backend configuration, used for storing task and queue state. Can select among different types. Currently, only local_dir is supported.

tasks:
  state_backend:
    type: local_dir
    path: example/state

Can be set by environment variables:

GITHUB_WATCHER_TASKS__STATE_BACKEND__TYPE=local_dir
GITHUB_WATCHER_TASKS__STATE_BACKEND__PATH=example/state

tasks.scheduler.limit - maximum number of parallel jobs. Default is 100.

tasks:
  scheduler:
    limit: 10

Can be set by GITHUB_WATCHER_TASKS__SCHEDULER__LIMIT environment variable.

tasks.scheduler.pending_limit - maximum number of pending jobs. 0 means no limit. Default is 0.

tasks:
  scheduler:
    pending_limit: 10

Can be set by GITHUB_WATCHER_TASKS__SCHEDULER__PENDING_LIMIT environment variable.

tasks.scheduler.timeout - maximum time for all jobs to finish in seconds. 0 means no timeout. Default is 600.

tasks:
  scheduler:
    timeout: 10

Can be set by GITHUB_WATCHER_TASKS__SCHEDULER__TIMEOUT environment variable.

tasks.scheduler.close_timeout - maximum time for scheduler to wait for all jobs to finish in seconds.

tasks:
  scheduler:
    close_timeout: 10

Can be set by GITHUB_WATCHER_TASKS__SCHEDULER__CLOSE_TIMEOUT environment variable.

Task Processors

Task (tasks.task_processor), trigger(tasks.trigger_processor) and event (tasks.event_processor) processors can be set by same block with the same structure.

tasks.[...]_processor.count - number of job processors. Default is 5.

tasks:
  [...]_processor:
    count: 10

Can be set by GITHUB_WATCHER_TASKS__[...]_PROCESSOR__COUNT environment variable.

tasks.[...]_processor.max_retries - maximum number of retries for failed jobs. Default is 3.

tasks:
  [...]_processor:
    max_retries: 5

Can be set by GITHUB_WATCHER_TASKS__[...]_PROCESSOR__MAX_RETRIES environment variable.

tasks.[...]_processor.queue_state_mode - queue state mode, sets queue state handling mode. Can be one of load, load_restart, accumulate and ignore. Default is load.

tasks:
  [...]_processor:
    queue_state_mode: restart

Can be set by GITHUB_WATCHER_TASKS__[...]_PROCESSOR__QUEUE_STATE_MODE environment variable.

tasks.[...]_processor.failed_queue_state_mode - failed queue state mode, sets queue state handling mode for failed jobs. Can be one of load, load_restart, accumulate and ignore. Default is accumulate.

tasks:
  [...]_processor:
    failed_queue_state_mode: ignore

Config

Config can be set by yaml file, example when using yaml_file config backend.

tasks:
  - id: ...
    type: ...
    triggers: ...
    actions: ...

Task config consists of next sections:

  • id - task id.
  • type - task type.
  • triggers - list of triggers. Currently, only github trigger is supported.
  • actions - list of actions. Currently, only telegram_webhook action is supported.

Once per run type

type: once_per_run - task will be executed only once per run.

tasks:
  - id: example_task
    type: once_per_run
    ...

Cron type

type: cron - task will be executed according to cron schedule.

tasks:
  - id: example_task
    type: cron
    cron: "0 0 * * *"
    ...
  • cron - cron schedule string. Use crontab.guru to generate cron schedule.

Development

Global dependencies

  • poetry

Taskfile commands

For all commands see Taskfile or task --list-all.