Split out of #101, which
recorded that "the Makefile is already stack-parameterised (STACK ?= observability), so the tooling supports it."
Half of that is true, and it is the half that does not matter. STACK ?=
reaches the lifecycle and secrets targets — up, down, render,
secrets-edit, reload. Every validator is pinned to the one directory:
Pinned to stacks/observability |
|
scripts/validate.sh:17 |
STACK="stacks/observability" |
scripts/check_docs.py:72 |
STACK = REPO / "stacks/observability" |
scripts/check_dashboards.py:37,38,56 |
dashboards, datasources, alloy dir |
scripts/check_compose_health.py:63 |
DEFAULT = ... |
scripts/check_loki_rules.sh:28 |
STACK= |
scripts/seed-validation-env.sh:26 |
STACK= |
.github/workflows/ci.yml:18 |
STACK: stacks/observability |
make check-rules is the exception that proves it: it uses $(STACK_DIR) and
so would follow STACK=lab, while make validate — which is what CI runs and
what a contributor runs — would not.
A stacks/lab/ created before this work is a stack CI has never seen. Its
compose is not config-checked, its rules are not promtool-tested, its
dashboards are not checked against its own datasources, its healthcheck
binaries are not probed, and its images are pinned and digest-verified by
nothing. Every guard this repository has built for the estate's stack would be
silently absent for the lab's — which is the shape of #62, #65, #68 and #79 all
over again, in a directory nobody is watching.
Worth deciding
- Iterate, or matrix? A
for stack in stacks/*/ inside validate.sh is
one change and keeps a single CI job. A GitHub Actions matrix gives a red
square per stack and a clearer failure, at the cost of the job list growing
with the estate.
- What is genuinely per-stack and what is repo-wide.
check_docs.py
asserts prose against configs and much of it is estate-specific — the SNMP
targets, the ports table, the compute table. Those are not "run it twice"
candidates; splitting repo-wide assertions from per-stack ones is most of the
work here.
- What a stack is allowed to lack. The lab has no snmp-exporter,
no blackbox-exporter and no Alertmanager (ADR-0019), so every checker needs
to distinguish "absent by design" from "missing". render-config.sh already
guards each render behind an if [[ -f ... ]]; its REQUIRED array does
not, and today demands all four SNMP communities and all four Alertmanager
URLs from any stack.
Done when
make validate and CI both check every directory under stacks/, and adding
an empty one fails loudly rather than passing quietly.
Split out of #101, which
recorded that "the
Makefileis already stack-parameterised (STACK ?= observability), so the tooling supports it."Half of that is true, and it is the half that does not matter.
STACK ?=reaches the lifecycle and secrets targets —
up,down,render,secrets-edit,reload. Every validator is pinned to the one directory:stacks/observabilityscripts/validate.sh:17STACK="stacks/observability"scripts/check_docs.py:72STACK = REPO / "stacks/observability"scripts/check_dashboards.py:37,38,56scripts/check_compose_health.py:63DEFAULT = ...scripts/check_loki_rules.sh:28STACK=scripts/seed-validation-env.sh:26STACK=.github/workflows/ci.yml:18STACK: stacks/observabilitymake check-rulesis the exception that proves it: it uses$(STACK_DIR)andso would follow
STACK=lab, whilemake validate— which is what CI runs andwhat a contributor runs — would not.
A
stacks/lab/created before this work is a stack CI has never seen. Itscompose is not
config-checked, its rules are notpromtool-tested, itsdashboards are not checked against its own datasources, its healthcheck
binaries are not probed, and its images are pinned and digest-verified by
nothing. Every guard this repository has built for the estate's stack would be
silently absent for the lab's — which is the shape of #62, #65, #68 and #79 all
over again, in a directory nobody is watching.
Worth deciding
for stack in stacks/*/insidevalidate.shisone change and keeps a single CI job. A GitHub Actions matrix gives a red
square per stack and a clearer failure, at the cost of the job list growing
with the estate.
check_docs.pyasserts prose against configs and much of it is estate-specific — the SNMP
targets, the ports table, the compute table. Those are not "run it twice"
candidates; splitting repo-wide assertions from per-stack ones is most of the
work here.
no blackbox-exporter and no Alertmanager (ADR-0019), so every checker needs
to distinguish "absent by design" from "missing".
render-config.shalreadyguards each render behind an
if [[ -f ... ]]; itsREQUIREDarray doesnot, and today demands all four SNMP communities and all four Alertmanager
URLs from any stack.
Done when
make validateand CI both check every directory understacks/, and addingan empty one fails loudly rather than passing quietly.