nickel-compose

Docker Compose is NOT composable. Nickel-Compose fixes that.

Docker Compose uses YAML, there is nothing good to be said about that. Nickel Compose provides a migration path to a real configuration language.

Nickel-driven compose: import existing YAML fragments, merge with Compose semantics, level up to real nickel config, export a single compose.yaml.

Why

Podman/Docker compose uses as YAML. Multi-fragment setups (root + services + overlays) usually combine fragments via a picker script or a colon-separated env var, but the merge semantics live in shell scripts and YAML quirks (!reset, anchor merge, ${VAR:?}). This project:

• replaces a picker with a single proper configuration config.ncl
• merges fragments in Nickel or Yaml with the same semantics Compose uses
• auto-fills defaults (networks, restart, init) so fragments stay small
• synthesizes top-level volumes: and networks: from service references, so that a root fragment is not needed
• supports conditional patches (if_present, if_absent) so a service fragment can adapt to the other services available
• validates your config against typed contracts — typos in service names, missing image, wrong port shapes all fail at typecheck time, not podman compose up time
• exports one compose.yaml that both podman-compose and docker compose auto-pick — no -f flag needed at deploy time

NICKEL_COMPOSE (optional path to the active config.ncl) is set by mise/CD-hook so that bare nickel-compose use resolves to the right config. Explicit use config.ncl always wins. Nickel-compose follows the convention .yml for input fragments and .yaml for the rendered whole.

Install

git clone https://github.com/keithy/nickel-compose.git
cd nickel-compose
mise trust         # trust mise/config.toml
mise install       # install nickel + jq

Usage

With mise tasks (recommended):

mise run check                       # typecheck the merge engine
mise run test                        # run all of the bash-spec test suites
mise run render                      # render examples/podclaws/config.ncl
mise run render -- config=path out=path   # render a custom config

Or directly:

./nickel-compose use examples/podclaws/config.ncl --out compose.yaml
nickel export --format yaml examples/podclaws/config.ncl > compose.yaml

See docs/testing.md for the spec/test suite and how to add tests. See docs/nickel-run.md for the nickel-run wrapper spec — a generic Nickel invocation tool that doubles as a proposal for a native nickel run subcommand (see nickel-lang/nickel#2636).

How it works

config.ncl  --[nickel export]-->  compose.yaml  --[podman compose]-->  containers
   |
   +-- imports YML/NCL fragments
   +-- applies defaults per service
   +-- merges fragments with Compose semantics

config.ncl is generated by `scripts/dc2nc.sh` from a curated list
of fragments, or hand-written directly.

The merge engine (nickel-compose.ncl) is a single Nickel file that exports a record of functions. The main one, merge, takes a list of fragments and returns a merged Compose record. config.ncl calls composer.merge with the fragments it has selected. Future helpers (validate, discover, etc.) can be added to the record without breaking callers.

Merge semantics

For each field in the later fragment (b):

Type of b	Type of a	Field in array_fields?	Action
scalar	scalar	n/a	b wins
array	array	yes	concat (a @ b)
array	array	no	b wins
record	record	n/a	recurse
anything	absent	n/a	insert from b

array_fields defaults to: environment, volumes, ports, extra_hosts, tmpfs, env_file, cap_add, cap_drop, security_opt.

services, volumes, networks are unioned across fragments — later wins on key collision.

Top-level synthesis

After merging, the engine scans every service's volumes and networks fields. Named volumes and networks referenced by services but not declared at the top level are synthesized as null body. Bind mounts (./path:, /abs:, ${VAR}:) are skipped. The default network is skipped (compose handles it implicitly). Any fragment can pre-declare a top-level entry with full config (driver, driver_opts, etc.) — pre-declared entries win over synthesis. So:

# services/web.ncl — declares a service that references web-data
{
  services = {
    web = { image = "nginx:1.27", volumes = ["web-data:/var/www/html"] },
  },
}

renders to:

services:
  web:
    volumes: ["web-data:/var/www/html"]
volumes:
  web-data: null   # synthesized from the service reference

To set NFS drivers or other options, add a top-level declaration to any fragment — including inline in services/web.ncl:

{
  services = { web = { volumes = ["web-data:/var/www/html"] } },
  volumes = { web-data = { driver = "local", driver_opts = { type = "nfs" } } },
}

Conditional patches

Fragments can declare patches that fire only when a specific service, volume, or network is (or isn't) selected. Two conditionals: if_present and if_absent. Each top-level key under them is a gate in the form "<field>::<value>" (double colon so gate values can contain dots). The value is a patch record that gets merged at the top level when the gate is met.

Both sides of :: accept regex patterns, not just literal names. * matches any chars, ? matches one char, and regex metachars in literal names (like the dot in my.app) are escaped automatically. So services::redis-.* matches redis, redis-cache, redis-sentinel, etc. The most common case — services::redis — works without any wildcards.

# dev.yml — patches web only when redis is selected
if_present:
  services::redis:
    services:
      web:
        environment: [REDIS_HOST=redis]
        depends_on: [redis]

# If you skip redis, web stays dependency-free.

# fallback.yml — provide a local DB when no external postgres
if_absent:
  services::postgres:
    services:
      db: { image: postgres:16-alpine }

Resolution order: if_absent first, then if_present. If both touch the same field, if_present wins (explicit presence beats default absence). Both fields are stripped from the rendered output — compose doesn't recognize them.

The gate field doesn't have to match the patch's first key. A volumes::home-data gate can patch services.web.environment, because the engine merges the patch at the top level via merge_records — the patch's structure determines where it lands.

Defaults

Each service gets these defaults filled in if missing:

{
  networks = ["default"],
  restart = "unless-stopped",
  init = false,
}

If a fragment already sets networks, that wins. Override the defaults in nickel-compose.ncl's default_service record.

Writing a config

let composer = import "../nickel-compose.ncl" in

let fragments = [
  import "./base.yml",
  import "./services/web.yml",
  import "./services/db.yml",
  import "./overlays/dev.yml",
] in

composer.merge fragments

A root fragment is optional. If your services reference named volumes and you don't need to set volume drivers or options, skip base.yml entirely — the merge engine synthesizes top-level declarations from service references.

What's not covered yet

• Cross-fragment validation — service.redis.yml references redis, but nothing enforces that another file declares it. composer.validation.dependencies will check this; queued for a later release. podman-compose catches it at up time for now.
• Non-array field overrides — Compose's ${VAR:?msg} is preserved through round-trip. The merge engine doesn't validate required envs.
• Comprehensive service schema — composer.Service contracts only the fields the engine consumes (image/build, command, environment, volumes, ports, depends_on, networks, restart, init). Everything else passes through untyped. A ServiceComprehensiveSchema is a future option if you want full Compose coverage.

See docs/schema.md for the schema design and roadmap.

Status

Verified end-to-end with nickel 1.17.0 and podman-compose 1.6.0. Test suite covers env concat, volume concat, default fill, full round-trip through podman-compose, schema contracts, the check function, the merge_fully_validate integration, and the nickel-compose-use.sh verb.

Layout

nickel-compose/
├── nickel-compose.ncl          # merge engine (single file, drop-in)
├── bin/
│   ├── nickel-compose           # dispatcher (execs bin/nickel-compose-<verb>.sh)
│   ├── nickel-compose-use.sh    # render config to compose.yaml
│   ├── nickel-compose-check.sh  # typecheck
│   ├── nickel-compose-report.sh # query the merged record
│   ├── nickel-compose-schema.sh # show a contract's fields
│   ├── nickel-compose-help.sh   # this message
│   ├── nickel-compose-run.sh    # thin wrapper: pre-loads the engine, calls nickel-run
│   ├── nickel-run.sh            # generic nickel invocation wrapper
│   └── nickel-compose-check.sh  # strict typecheck (engine + optional user config)
├── scripts/
│   └── dc2nc.sh                 # fragment picker (stdin or --find-all) → bare-list config.ncl
├── examples/
│   ├── dummy-project/          # self-contained first-time-user example
│   └── podclaws/               # example using real podclaws fragments
├── tests/
│   ├── merge.ncl               # synthetic merge fixture
│   ├── *_spec.sh               # one bash-spec file per context
│   ├── lib/
│   │   └── bash-spec.sh        # vendored bash-spec 2.1
│   ├── out/                    # rendered outputs (gitignored)
│   ├── expected/               # golden snapshots (committed)
│   └── fixtures/               # synthetic fragments for engine tests
├── docs/
│   ├── design.md               # design rationale
│   ├── workflow.md             # central workflow + migration paths
│   ├── schema.md               # contracts, check, merge_fully_validate
│   ├── nickel-run.md           # nickel-run wrapper spec (proposed upstream API)
│   └── testing.md              # test suite, golden-file testing
├── mise/
│   ├── config.toml             # tools (nickel, jq) + task config
│   └── tasks/
│       ├── check               # typecheck the engine and user config
│       ├── render              # render config to compose.yaml
│       └── test                # run the bash-spec test suite
├── README.md
├── LICENSE
└── .gitignore

The dispatcher (bin/nickel-compose) shells out to bin/nickel-compose-<verb>.sh; verb scripts call bin/nickel-compose-run.sh for the merge, which calls bin/nickel-run.sh for the eval. Domain helpers (dc2nc.sh fragment picker) live in scripts/. Don't move a script between bin/ and scripts/ without also updating the tests that reference it by path.

License

MIT — see LICENSE.