Skip to content

Make the preflight work on the server stack, where it was both wrong and unsafe - #6

Merged
tschm merged 1 commit into
mainfrom
fix/preflight-works-on-server
Aug 29, 2026
Merged

Make the preflight work on the server stack, where it was both wrong and unsafe#6
tschm merged 1 commit into
mainfrom
fix/preflight-works-on-server

Conversation

@tschm

@tschm tschm commented Aug 29, 2026

Copy link
Copy Markdown
Member

check-public-safe.sh was written against the laptop stack. Run against a real deployment it gave four wrong answers, including a false PASS.

Actual output from the first production run:

PASS  all 1 exported repos are public on GitHub
FAIL  anonymous /api/ds/query returned HTTP 000 - a visitor can run any PromQL
FAIL  anonymous datasource proxy returned HTTP 000 - the label index is readable
PASS  jq-prometheus is not published beyond loopback
WARN  JQ_PUBLIC_ONLY is '' - fine for local use

Every line of that is wrong. The deployment was in fact correct — verified independently from outside: /api/search 401, datasource proxy 401, POST /api/ds/query 401, ports 3000/9090/9109 all closed, valid Let's Encrypt certificate.

The four defects

False FAIL ×2 Probed http://localhost:3000, which the TLS overlay deliberately doesn't publish. Got HTTP 000 and reported "a visitor can run any PromQL" — the opposite of the truth.
Vacuous PASS Read the collector's metrics from localhost:9109, unpublished on a server. The empty result made wc -l report 1, so it announced "all 1 exported repos are public" having checked nothing.
Dangerous PASS Looked for containers literally named jq-prometheus. The server stack names them jq-monitoring-prometheus-1, so docker inspect failed, the grep for 0.0.0.0 found nothing, and it passed — it would have passed identically with the port wide open.
Spurious WARN Read JQ_PUBLIC_ONLY from .env; only the laptop sets it there, the server stack fixes it in the compose file.

The fix

  • Grafana is probed where it actually is — the public HTTPS URL when FLEET_DOMAIN is set, localhost otherwise.
  • Refused and unreachable both count as safe; only a 2xx is a failure.
  • The collector is asked for its metrics from inside its own container.
  • Containers are resolved by compose project label, and a missing container is now a failure, not a pass.
  • JQ_PUBLIC_ONLY is read from the running container's environment.

Verified

Against the live laptop stack: 23 repos enumerated and each checked against GitHub, anonymous access correctly failed (that stack genuinely isn't safe to expose), loopback-published ports correctly warned rather than silently passed.

🤖 Generated with Claude Code

…and unsafe

The check was written against the laptop stack and gave four wrong answers on a
real deployment - including a false PASS, which is the dangerous kind.

- It probed http://localhost:3000, which the TLS overlay deliberately does not
  publish, got HTTP 000, and reported "a visitor can run any PromQL". Refused
  and unreachable are now both treated as safe; only a 2xx is a failure.
- It read the collector's metrics from localhost:9109, which the server stack
  does not publish either. The empty result made `wc -l` report 1, so it
  announced "all 1 exported repos are public" having verified nothing. It now
  asks the collector from inside its own container.
- It looked for containers literally named jq-prometheus and jq-collector. The
  server stack names them jq-monitoring-prometheus-1, so `docker inspect`
  failed, the grep for 0.0.0.0 found nothing, and it PASSED - it would have
  passed just as happily with the port wide open. Containers are now resolved
  by compose project label, and a missing container is a failure rather than a
  pass.
- It read JQ_PUBLIC_ONLY from .env, which only the laptop sets; the server
  stack fixes it in the compose file. It now reads the running container's
  environment.

Verified against the live laptop stack: 23 repos enumerated and checked against
GitHub, anonymous access correctly failed, loopback-published ports correctly
warned rather than silently passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@tschm
tschm merged commit e48c444 into main Aug 29, 2026
4 checks passed
@tschm
tschm deleted the fix/preflight-works-on-server branch August 30, 2026 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant