Skip to content

v0.12.3 — audit-backlog health signal & degrade-safe hardening

Choose a tag to compare

@dberry37388 dberry37388 released this 22 Jun 01:44
8680201

CTO review follow-ups on the v0.12.2 tree — truthful audit-backlog health signal and degrade-safe failure-path hardening.

Fixed

  • swarm:health now ages unclaimed audit-outbox rows, closing a false "relay appears active" signal (review F1). runAuditOutboxStalenessCheck() only warned on pending rows whose reservation had expired (reserved_at set and stale). Rows the relay had never claimed (reserved_at IS NULL) were never aged, so a relay that is unscheduled, misrouted, or starved before claim let audit evidence back up while the check still reported "<n> pending row(s), relay appears active" — a false clean signal in a regulated deployment where audit-backlog observability is a compliance concern, not an ops nicety. The check now also counts unclaimed pending rows older than the warning threshold (by created_at) and reports the two backlog modes separately in the warning detail (<n> pending row(s) with stale reservations vs <n> unclaimed pending row(s) aging past <T>s). The threshold reuses the existing swarm.durable.relay.stale_warning_threshold_seconds key (the audit outbox already shares the durable relay timeout), so no new configuration surface is introduced; the config and docs/configuration.md entries for that key now document that it governs both the durable (swarm:health --durable) and audit (bare swarm:health) staleness checks. Recent unclaimed rows still read ok (normal between relay runs). Mirrors the existing, proven runOutboxStalenessCheck() durable-outbox pattern. The health check's staleness/warning thresholds are now computed in UTC (now('UTC')) to match how the outbox stores created_at/reserved_at (Carbon::now('UTC')) — a bare now() compared app-timezone wall-clock against the UTC columns and skewed the boundary by the UTC offset on any non-UTC app.timezone, hiding (or prematurely flagging) genuinely-aged rows; this corrects the audit and durable staleness checks. Covered by a non-UTC-timezone regression test.

  • The audit/durable "degrade safely" paths can no longer become a second failure surface if the logging or error-reporting stack itself throws (review F2). The fallback report() and logger->{level}() calls on the failure-handling paths assumed the logging/reporting stack was safe; a hostile or misconfigured logger could turn a contained failure into an uncaught one. A new internal BuiltByBerry\LaravelSwarm\Support\SafeReporting trait centralizes containment behind never-throw safeReport() / safeLog() helpers (with an error_log() last resort that is itself guarded), and is now used by ConfiguredSinkFailureHandler (all five policy log calls), DatabaseAuditOutbox (the three report() calls plus the dead-letter transition log), and DatabaseDurableOutbox (both drain-loop report() calls). The failure_policy=halt decision still returns Halt (and the dispatcher still throws) when the logger throws — only the log/report call is contained, never the halt itself. safeLog() invokes the level-specific logger method, so logging behavior is byte-identical to before; only the throw is contained. The error_log() last resort emits only the exception class (never getMessage()), so the rare double-failure breadcrumb cannot leak a caller-supplied audit payload into PHP's ungoverned system error log — the full detail is still persisted sealed in the outbox row's last_error. Covered by a new hostile-logger/hostile-reporter test suite exercising the trait in isolation, the error_log leak boundary (asserting the class is present and a sensitive sentinel is absent), the failure handler (including halt-survives-hostile-logger), and the real DatabaseAuditOutbox::drain() dead-letter path with both the sink and the logger throwing.

    Operator-facing notes for the swarm:health output change and the timezone correction are in UPGRADING.md (exit code and --json ok are unchanged).