From 5926ecaa9063f836db4d08cffe668098f36f8413 Mon Sep 17 00:00:00 2001 From: Garrett Allen Date: Sun, 2 Aug 2026 21:17:49 +0000 Subject: [PATCH 1/2] fix(secrets): make `make secrets-init` actually work MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reported from a real run on the monitoring host: -- creating observability.sops.yaml from the template error loading config: no matching creation rules found make: *** [Makefile:62: secrets-init] Error 1 SOPS selects a creation_rule by matching path_regex against the *input* path. The script encrypted the template and redirected the output: sops --encrypt secrets/observability.example.yaml > secrets/observability.sops.yaml so SOPS tested `secrets/.*\.sops\.ya?ml$` against the **example** filename, which does not end in .sops.yaml. No rule matched. The output name is never consulted, so the destination being correct made no difference. Copy to the destination name first, then encrypt in place, which is the path the rule actually targets. Two guards added, because the failure mode of getting this wrong is worse than an error message. `cp` followed by a failed encrypt leaves a plaintext file sitting at a path whose name says "encrypted" — and it is not gitignored, because encrypted secrets are meant to be committed. So on any encryption failure the partial file is removed, and success is confirmed by checking for the `sops:` metadata block rather than trusting the exit status. CI catches a plaintext secrets file, but only after it has been pushed. This was never caught because the whole path needs an age key, and the earlier work stopped at "cannot verify without the user's key" rather than generating a throwaway one. Verified end to end in a sandbox with its own HOME and a disposable keypair: 1. bootstrap produces a genuinely encrypted file — keys readable, values ENC[AES256_GCM...], no placeholder text remaining 2. `sops --encrypt --in-place` round-trips after editing values 3. render-config.sh writes snmp.yaml, webhook_url and .env 4. a community containing / & \ and $ renders byte-for-byte intact, and all four devices keep distinct values 5. no SNMP community reaches .env — only the two values compose interpolates 6. `docker compose config` passes against the real rendered .env 7. git sees only .sops.yaml and the encrypted secrets file as stageable; no .env, .rendered/ or plaintext copy can be committed --- scripts/bootstrap.sh | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/scripts/bootstrap.sh b/scripts/bootstrap.sh index 9135658..27fdca2 100755 --- a/scripts/bootstrap.sh +++ b/scripts/bootstrap.sh @@ -68,8 +68,33 @@ if [[ -f "${SECRETS_FILE}" ]]; then info "$(basename "${SECRETS_FILE}") already exists — leaving it alone" else info "creating $(basename "${SECRETS_FILE}") from the template" - sops --encrypt "${EXAMPLE_FILE}" > "${SECRETS_FILE}" + + # Copy to the destination name FIRST, then encrypt in place. + # + # SOPS chooses a creation_rule by matching path_regex against the *input* + # path. Encrypting the template directly — `sops -e observability.example.yaml + # > observability.sops.yaml` — makes SOPS test the rule against the example + # filename, which does not end in .sops.yaml, so no rule matches and it exits + # with "no matching creation rules found". The output name is never consulted. + cp "${EXAMPLE_FILE}" "${SECRETS_FILE}" chmod 600 "${SECRETS_FILE}" + + if ! sops --encrypt --in-place "${SECRETS_FILE}"; then + rm -f "${SECRETS_FILE}" + die "encryption failed — removed the partial file rather than leave +plaintext credentials sitting at a path that looks encrypted. + +Check that .sops.yaml lists a valid age recipient: + grep -A2 creation_rules ${SOPS_CONFIG}" + fi + + # A plaintext file at this path would be committed as if it were encrypted, + # and CI only catches that after the push. Verify before claiming success. + if ! grep -q '^sops:' "${SECRETS_FILE}"; then + rm -f "${SECRETS_FILE}" + die "sops reported success but produced no encrypted output — file removed" + fi + warn "It still contains the placeholder values. Edit it now:" warn " make secrets-edit" fi From 46bb17296948e28f0d510c0a3097c7e7367d13bf Mon Sep 17 00:00:00 2001 From: Garrett Allen Date: Sun, 2 Aug 2026 21:24:47 +0000 Subject: [PATCH 2/2] fix(secrets): guard the plaintext window with a trap, not just two checks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review feedback, and correct: cleanup only covered the two failures explicitly checked. Between the `cp` and a verified encryption the file is plaintext at a path that is meant to be committed and whose name says "encrypted" — and it is deliberately not gitignored, so anything leaving it behind is a leak waiting to be committed. A chmod failure under `set -e`, a Ctrl-C mid-encrypt, or a SIGTERM all skipped the rm. A trap on EXIT INT TERM is now armed immediately after the cp and cleared only once the sops metadata block is confirmed present. Verified in an isolated sandbox with its own HOME and a disposable keypair: * happy path still produces an encrypted file * a broken age recipient leaves nothing behind * SIGINT delivered to the script's own process group mid-encrypt — a real Ctrl-C — leaves nothing behind: present before the signal, absent after Three earlier attempts at that last test were inconclusive rather than passing, which is worth recording because each one looked like a result: * signalling a process group that, in a non-interactive shell, also contained the test harness * sending SIGINT to the script's pid, where bash defers the handler until the foreground command returns — sops completed and the file legitimately survived encrypted, which reads as a leak unless you check the contents * a setsid run whose $! had already exited, so the kill addressed an empty pgid and no signal was ever sent Only the setsid + pgrep-resolved pgid version actually exercises the path. --- scripts/bootstrap.sh | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/scripts/bootstrap.sh b/scripts/bootstrap.sh index 27fdca2..66441d9 100755 --- a/scripts/bootstrap.sh +++ b/scripts/bootstrap.sh @@ -77,24 +77,36 @@ else # filename, which does not end in .sops.yaml, so no rule matches and it exits # with "no matching creation rules found". The output name is never consulted. cp "${EXAMPLE_FILE}" "${SECRETS_FILE}" + + # From here until encryption is verified, the file is PLAINTEXT sitting at a + # path that is meant to be committed and whose name says "encrypted". It is + # deliberately not gitignored, so anything that leaves it behind is a leak + # waiting to be committed. + # + # The trap covers every exit, not just the two failures checked below: a + # chmod failure under `set -e`, a Ctrl-C mid-encrypt, a SIGTERM. Cleared only + # once the sops metadata block is confirmed present. + trap 'rm -f "${SECRETS_FILE}"' EXIT INT TERM + chmod 600 "${SECRETS_FILE}" if ! sops --encrypt --in-place "${SECRETS_FILE}"; then - rm -f "${SECRETS_FILE}" - die "encryption failed — removed the partial file rather than leave -plaintext credentials sitting at a path that looks encrypted. + die "encryption failed — the partial file has been removed rather than +leave plaintext credentials at a path that looks encrypted. Check that .sops.yaml lists a valid age recipient: grep -A2 creation_rules ${SOPS_CONFIG}" fi - # A plaintext file at this path would be committed as if it were encrypted, - # and CI only catches that after the push. Verify before claiming success. + # Do not trust the exit status alone. CI catches a plaintext secrets file, but + # only after it has been pushed. if ! grep -q '^sops:' "${SECRETS_FILE}"; then - rm -f "${SECRETS_FILE}" die "sops reported success but produced no encrypted output — file removed" fi + # Confirmed encrypted: the file may now survive. + trap - EXIT INT TERM + warn "It still contains the placeholder values. Edit it now:" warn " make secrets-edit" fi