Configuration

Configuration file can be specified by -c {config.yaml}.

Example

interval: 3 # Interval between pollings
loglevel: 5 # 0-5. 0 for ERROR and 5 for DEBUG
concurrent_limit: 3 # number of workers that can run at the same time
# Prometheus metrics are exposed at http://exporter_address/metrics
exporter_address: :8081

dummy: # you can insert anything here. The main use is to define yaml anchors
    common: &common_interval
        interval: 3600 # Interval between sync

#logstash:
#   address: listener.logz.io:5050 # logstash sink. Lug will send all logs to this address
#   additional_fields:
#       token: "" # Additional fields sent to logstash server

# Address where JSON API will be served
json_api:
    address: :7001

logstashaddr: "172.0.0.4:6000" # TCP Address of logstash. empty means no logstash support
repos:
    - type: rsync # Config for repo1
      source: rsync://rsync.chiark.greenend.org.uk/ftp/users/sgtatham/putty-website-mirror/
      <<: *common_interval # will insert interval: 3600 here. 3600s between syncs
      path: /tmp/putty # Output directory
      name: putty # Required
      rlimit_mem: 200M # Optional, maximum memory can be used
      retry: 3 # Numbers of retry when failed, by default 3 (ver 0.12+). Only valid for some worker
      retry_interval: 3 # Seconds between retry, by default 3 (ver 0.12+). Only valid for some worker
# You can add more repos here, different repos may have different worker types,
# refer to Worker Types section for detailed explanation

Worker Types

~~~Rsync~~~

**Since 0.10, rsync worker is moved out of this project. Users may want to use rsync.sh with shell_script worker. **

type: rsync # required
source: rsync://rsync.chiark.greenend.org.uk/ftp/users/sgtatham/putty-website-mirror/ # with rsync:// prefix
interval: 6 # Interval between sync in seconds
path: /tmp/putty # Output directory
name: putty # Required
# exclude_hidden: true # Exclude hidden files from upstream. By default this is not enabled

Shell Script

All config items will be forwarded to the script as environmental variables, with an additional LUG_ prefix.

type: shell_script # required
script: /path/to/your/script # required
interval: 6 # Interval between sync in seconds
name: shell # Required

External

External worker is a special type of worker. It simply does nothing and always returns {Idle: true, Result: true}. Utilities will not be executed. Any key except name/type will be ignored.

name: external_worker
type: external
any_key: any_value

Note that this could be used to guide gen_caddyfile.sh to generate reverse proxy.

Utilities

Utilities are functions that work on multiple kinds of workers.

Rlimit

Add

    rlimit_mem: 200M

in config of each repo to limit the memory that worker can use.

JSON Restful API

By default lug listens on address :7001, which could be overwritten by -j ADDR.

If both -cert and -key are provided, HTTPS instead of HTTP protocol will be used.

rest.Get("/lug/v1/manager", r.getManagerStatus),     // return manager.Status object
rest.Get("/lug/v1/manager/summary", r.getManagerSummary),     // return manager.Status object without stderr/stdout
rest.Post("/lug/v1/admin/manager/start", r.startManager),  // no return value
rest.Post("/lug/v1/admin/manager/stop", r.stopManager),    // no return value
rest.Delete("/lug/v1/admin/manager", r.exitManager),       // no return value

Currently all request has no required param.

Since no authentication is required, we suggest running this API behind a nginx reverse proxy with authentication method.