v0.5.1-beta
Spooky 0.5.1-beta — Observability Operator Bundle
Patch release covering all changes since 0.5.0-beta. Ships a packaged operator observability experience — dashboards, alerts, and SLOs on top of the existing metrics — plus canonical audit correlation, hardened default packaging, and an OTLP startup fix.
Added
- Packaged operator observability bundle under
deploy/observability/— Prometheus recording rules, production alert rules, an SLO package, and six Grafana dashboards (edge traffic, admission overload, backend health, retries/hedges, TLS certificates, control-plane activity), so operators get a working dashboard/alert set instead of assembling one from raw metrics. - Canonical correlation fields on admin audit events —
event_id,schema_version,request_id,trace_id,span_id,listener— plus a stablefailure_class(authentication,authorization,source_policy,request_validation,runtime_config,runtime_state,listener_tls,watchdog) on non-success events, so failures can be grouped and correlated across metrics/logs/traces without parsing free-form reason strings. GET /admin/runtimeand the runtime history endpoints gained anobservabilityblock — contract version, audit schema version, current generation, backend and quota backend health summaries, recent tracked admin actions, and repository-relative dashboard/documentation references. This is the one canonical entry point into the packaged bundle for operators and automation.h3_clientnow accepts--method(defaultGET) and repeatable--header name=value(including pseudo-headers like:protocol), for exercising non-GET and header-sensitive traffic in the observability lab.- Documentation —
docs/operations/observability-bundle.mdcovers the shipped dashboards, alerts, SLOs, and incident-correlation workflow;docs/architecture/observability-contract.mdanddocs/operations/control-plane.mddocument the audit schema and the new runtimeobservabilityblock.
Changed
- The Debian package's default
config.yamlnow ships the control API disabled (observability.control_api.enabled: false) with a placeholder-token comment block, instead of enabled with a literalreplace-with-strong-tokencredential — a fresh install no longer boots with a live, weakly-credentialed admin surface. - The systemd unit restarts with
Restart=alwaysinstead ofRestart=on-failure(the watchdog's own clean-exit drain would otherwise not be restarted), addsStartLimitIntervalSec=60/StartLimitBurst=5to guard against crash-looping, raisesLimitNOFILE=65535for QUIC-scale file descriptor use, and tightens sandboxing (ProtectKernelTunables,ProtectKernelModules,ProtectControlGroups,RestrictAddressFamilies,RestrictNamespaces,RestrictSUIDSGID,LockPersonality)./etc/spookyis no longer inReadWritePaths— only/var/log/spookystays writable.
Fixed
- OTLP tracing is now initialized inside the Tokio runtime instead of before it. The OTLP tonic exporter spawns a background task while building its gRPC channel, which previously panicked with "there is no reactor running" whenever tracing was enabled.
- Docker image build and clippy warnings introduced during observability bundle development.
Upgrade Notes
Nothing breaks. The new observability block on /admin/runtime and the audit event fields are additive JSON — existing consumers that don't read them are unaffected.
enabled: true.
Existing deployed configs and units are not modified in place by this release — these changes affect newly generated packages only.
Known Limitations
- Observability contract and audit schema are both fixed at v1 — no versioned migration path yet if the schema needs to change.
- Dashboard/documentation references in
/admin/runtimeare repository asset paths, not resolvable URLs — operators must map them into their own Grafana/runbook tooling. - Pre-GA — soak test before broad rollout.