Skip to content

Includes and Defaults

Jump to bottom
ptweezy edited this page Jun 23, 2026 · 3 revisions

Includes, Defaults, and Multi-File Config

This page documents how yacron2 loads configuration: the -c argument as a single file or a directory, the defaults section and its merge precedence, the include directive, and the special list-merge rules that govern how job settings inherit from defaults. All behavior here is implemented in yacron2/config.py.

Config loading entry points

yacron2 resolves the -c/--config argument (default /etc/yacron2.d on POSIX, %APPDATA%\yacron2 on Windows — falling back to ~ if APPDATA is unset; see CLI Reference and Running on Windows) through parse_config(config_arg):

  • If config_arg is a directory, it is loaded by _parse_config_dir (directory mode, below).
  • Otherwise it is loaded as a single file by parse_config_file. parse_config wraps that call in a try/except OSError, so a missing or unreadable file (OSError) is re-raised as a ConfigError and the CLI reports it cleanly rather than surfacing a bare OSError.

A single file is read as UTF-8, validated against CONFIG_SCHEMA with strictyaml, and parsed by parse_config_string. The top level accepts an empty document (EmptyDict()) or a mapping with the optional keys defaults, jobs, web, include, and logging (all Opt(...) — none is required). See the Configuration Reference for the per-job and web schemas, and Logging Configuration for the logging schema.

Directory mode

When -c points at a directory, _parse_config_dir enumerates the directory's entries with os.scandir and processes them in sorted filename order (sorting makes job ordering and "first config found" error messages deterministic rather than dependent on filesystem order — a fix new in 1.0.4).

For each entry, the name is split into base and extension:

  • The entry is skipped if the first character of its base name is _ or .. This lets you keep shared include fragments (e.g. _inc.yaml) and dotted files in the same directory without them being loaded as standalone configs.
  • The entry is skipped if its extension is anything other than .yml or .yaml.

Each remaining file is parsed independently with parse_config_file, then its results are aggregated across the directory:

Aggregate Behavior across files
jobs Concatenated in sorted-filename order (jobs.extend(...)).
defaults Merged across files via mergedicts into a single directory-wide job_defaults (used only as the returned job_defaults; see the caveat below).
web At most one. A second file containing a web block raises ConfigError("Multiple 'web' configurations found: first in <file>, now in <file>").
logging At most one. A second file containing a logging block raises ConfigError("Multiple 'logging' configurations found: ...").

Per-file parse errors are collected (keyed by path) and, if any occurred, raised together as a single ConfigError whose message joins the individual errors with \n---. An empty directory, or one where every entry is skipped, yields an empty Yacron2Config (empty jobs, no web, empty job_defaults, no logging) rather than an error. Aggregating jobs/defaults/logging across all files (instead of using only the last file's settings), rejecting duplicate web/logging blocks, and the empty-directory result are all behaviors new in 1.0.4.

Defaults are scoped per YAML file in directory mode

This is the most important caveat of directory mode: the defaults section in each file applies only to the jobs defined in that same file. Each file is parsed by its own parse_config_string call, where that file's defaults are merged into its own jobs before the files are aggregated. The directory-wide job_defaults returned by _parse_config_dir is the merge of every file's defaults, but it is not applied retroactively to jobs that were already constructed in other files. To share defaults across files in a directory, put them in an include fragment and include it from each file (see below).

The defaults section

A defaults mapping supplies inherited values for the jobs defined alongside it. It accepts the same options as a job (_job_defaults_common) except for the job-only required keys name, command, and schedule. Every option in defaults is optional. Jobs may override any inherited value.

defaults:
  shell: /bin/bash
  utc: false
  environment:
    - key: PATH
      value: /bin:/usr/bin
jobs:
  - name: test-01
    command: echo "foobar"   # inherits /bin/bash, utc:false, PATH
    schedule: "*/5 * * * *"
  - name: test-02
    command: echo "zbr"
    shell: /bin/sh           # overrides the inherited shell
    schedule: "*/5 * * * *"

The shell field works on every OS; the /bin/... paths above are POSIX examples. On Windows you'd set e.g. shell: powershell, leave it empty to use cmd.exe (via %ComSpec%), or pass command as a list to bypass the shell entirely (see Running on Windows).

Merge precedence

Within a single parsed file (parse_config_string), each job's effective configuration is built by successively merging with mergedicts, in this order (later wins):

  1. DEFAULT_CONFIG — the built-in defaults (e.g. shell: /bin/sh on POSIX (empty on Windows, which routes a string command through %ComSpec%/cmd.exe; see Running on Windows), captureStderr: true, utc: true, killTimeout: 30; full list in the Configuration Reference).
  2. Included files' defaults — the defaults blocks of any files named by this file's include directive, merged together in include order.
  3. This file's defaults block.
  4. Per-job overrides — the keys set on the individual job.

Each job dict is mergedicts(defaults, config_job), where defaults is mergedicts(mergedicts(DEFAULT_CONFIG, included_defaults), this_files_defaults). The resulting per-job dict is then passed to JobConfig, which validates it (numeric ranges, timezone, user/group). On Windows, user/group switching is unsupported: a job with user or group set raises a configuration error (Job <name>: changing user/group is not supported on Windows); see Running on Windows. The merged defaults (steps 1–3) is also returned as the file's job_defaults.

Merge semantics (mergedicts)

mergedicts(dict1, dict2) is a recursive deep merge where, on key collision, dict2 takes precedence. Its rules:

Case Behavior
Both values are mappings Recurse (deep merge).
dict1 value is a mapping, dict2 value is None Keep dict1's mapping (treat None as "not overridden").
Both values are lists, key is environment Merge by key (see below).
Both values are lists, key is fingerprint Replace with dict2's list (no concatenation).
Both values are lists, any other key Concatenate (v1 + v2).
Otherwise (scalars, type mismatch) Take dict2's value.
Key only in dict1 Keep dict1's value.
Key only in dict2 Keep dict2's value.

environment merges by key

environment is a list of {key, value} mappings. When a job and a default both define environment, they are merged into a dictionary keyed by key (default entries first, then the job's), so a job's variable overrides the default with the same name instead of producing two list entries with the same key. This is a behavior change in 1.0.0 (previously the lists were concatenated, yielding duplicate-keyed entries). See Commands and Environment for environment and env_file.

defaults:
  environment:
    - key: FOO
      value: foo
    - key: BAR
      value: bar
jobs:
  - name: test-01
    command: env
    schedule: "* * * * *"
    environment:
      - key: FOO        # overrides the default FOO -> single FOO=xpto entry
        value: xpto
      - key: ZBR        # added alongside the inherited BAR
        value: blah

The job above runs with FOO=xpto, BAR=bar, ZBR=blah.

fingerprint replaces, does not append

The Sentry fingerprint (a list of strings, default ["yacron2", "{{ environment.HOSTNAME }}", "{{ name }}"]) is a replace-not-append setting: a job or defaults block that supplies its own fingerprint overrides the default list entirely. Plain list concatenation would silently prepend the three default entries, making custom Sentry issue grouping impossible. This replace behavior is a fix new in 1.0.4. See Reporting for the Sentry reporter.

All other list-valued options concatenate.

The include directive

include is an optional list of file paths (new in 0.13). Each path is resolved relative to the directory of the including file (os.path.join(os.path.dirname(path), include)). Included files are parsed recursively with parse_config_file, and their results are merged into the including file as follows:

  • Jobs from included files are appended to this file's job list. Crucially, these jobs arrive already fully constructed by the included file's own parse_config_string, so they carry only their own file's defaults (and DEFAULT_CONFIG). A defaults block in the including file does not retro-apply to jobs that came from an included file.
  • Defaults from included files are merged together (in include order) into inc_defaults_merged, which is folded into the merge precedence at step 2 above — i.e. included defaults affect only this file's inline jobs, not the included files' own jobs.
  • web from an included file is adopted if this file has none; if both define web, parsing raises ConfigError("multiple web configs").
  • logging from an included file is adopted if this file has none; if both define logging, parsing raises ConfigError("multiple logging configs").

The intended use is to put common definitions (reporting defaults, shell, environment, etc.) in a fragment named so directory mode skips it (e.g. with a leading underscore), and include it from each real config file. This mirrors example/adhoc.yacron2.d:

_inc.yaml (skipped by directory mode; provides shared defaults, web, and logging):

defaults:
  shell: /bin/bash
  concurrencyPolicy: Allow
  environment:
    - key: FOO
      value: foo
    - key: BAR
      value: bar

test.yaml (a real config file that includes the fragment):

include:
  - _inc.yaml
jobs:
  - name: test-01
    command: echo "foobar"   # inherits shell:/bin/bash and FOO/BAR from _inc.yaml
    schedule: "@reboot"
    captureStdout: true

Because _inc.yaml contains only defaults (no jobs), its defaults flow into test.yaml's inline jobs through the include-defaults merge. If _inc.yaml also defined jobs, those jobs would carry _inc.yaml's own defaults and would not pick up test.yaml's defaults.

Include cycle detection

parse_config_file tracks visited files by absolute path in a _seen set that is threaded through the recursive include parse. A file that includes itself, directly or transitively, raises a clear ConfigError("include cycle detected at <path>") instead of recursing until a RecursionError (a fix new in 1.0.4). The _seen set is scoped to a single top-level parse_config_file call (it starts empty), so two independent files that both include a common fragment are not flagged as a cycle — diamond includes are allowed; only true cycles are rejected.

In directory mode, _seen is not shared across the directory's files: each file in the directory is parsed with its own fresh cycle-detection state.

Related pages

This wiki documents yacron2. See the README and the changelog.

yacron2 is a fork of gjcarneiro/yacron.

yacron2 Wiki

Clone this wiki locally