Promote observability metrics wave to main (GT-542/543/546/550/549 · board 515/551)#186
Merged
Conversation
…olith Core) Register the Evolith Core (Node-side) observability wave that turns the already-approved ADR-0007/ADR-0028 OSS stack operational. Producers already exist (/metrics on core-api/mcp/agent-runtime, OTel on all four processes) and the backend is scaffolded in the ops compose; this wave closes the gap between built producers and a wired, visualized, alerting backend. - GT-542 (P1): flagship gate metric declared but never emitted - GT-543 (P1): SLO/alert PromQL references metrics the code doesn't emit - GT-544 (P2): Grafana has no Prometheus datasource and zero dashboards - GT-545 (P2): scrape config covers only core-api at a wrong target - GT-546 (P1): Agent Runtime exposes only default Node metrics - GT-547 (P2): obs backend runs only in the UMS template compose - GT-548 (P2): business metrics carry no tenant dimension - GT-549 (P2): /metrics public on agent-runtime/mcp vs guarded on core-api - GT-550 (P2): no anti-drift guard for metrics cited in alerts/SLO - GT-551 (P2): OPA sidecar metrics expected by alerts not exposed/scraped Scope: Evolith Core only — Tracker/UMS/MMS/.NET items belong to their repos' boards. Board EN/ES parity 525/525; counter 510/551 · 20 pending. Tracking guard green (551 gaps / 492 closure records). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… prep) prepack removes the incremental tsc build cache before packing so the published @beyondnet/evolith-cli tarball no longer ships dist/.tsbuildinfo (310KB). Clean re-do on develop (the earlier commit accidentally captured another session staged files in the shared index during a multi-session collision). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…; reconcile alerts/SLO; drift guard (GT-542/543/546/550)
Turns the built-but-inert metrics plane operational — producers existed, but the
flagship signals were never emitted and the alert/SLO layer queried names no code
emits, so the alerts were dead.
GT-542 — the flagship gate signal was DECLARED but never emitted. MetricsService gains
recordGateEvaluation(gateId, verdict, phase, dur); wired into EvaluationController
(inline/canonical/legacy paths) and GatesController so evolith_gate_evaluations_total
{status,gateId,phase} + evolith_gate_evaluation_duration_seconds now record.
GT-543 — added the evolith_http_request_duration_seconds histogram (none existed → the
HighLatency alert + latency SLOs were inert), recorded from the SecurityAudit
interceptor; reconciled prometheus-alerts.yml + core-api-slo.md/.es.md from bare
http_requests_total / http_request_duration_seconds_bucket to the evolith_-prefixed
names the code actually emits.
GT-546 — AgentMetricsService adds the AI-era execution signal on the default registry
(so the existing /metrics serves it): evolith_agent_runs_total{engine,verdict},
evolith_agent_run_duration_seconds, evolith_skill_invocations_total{skill},
evolith_agent_core_calls_total{outcome}, evolith_hitl_approvals_total{decision},
recorded from the runtime result via the controller. Derivations are conservative —
only what the AgentRuntimeResult/trace attests.
GT-550 — metric-drift.spec.ts fails the PR when an alert/SLO references an Evolith metric
no service emits (the root cause of GT-543/545), with an allowlist for external
(node/kube/rabbitmq/opa) metrics.
Verified: core-api 150/150 (+10), agent-runtime-api 72/72 (+6), both tsc clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rd 514/551) Closes the 4 code-actionable observability gaps implemented in 56028e5: gate metric emitted (GT-542), HTTP-latency histogram + alert/SLO reconciliation (GT-543), agent-runtime execution metrics (GT-546), metric drift guard (GT-550). Board 510→514 done · 20→16 pending. Guard 08 green (551 gaps / 496 records). GT-546 dependency on the (non-existent) GT-519 metrics port dispositioned accepted-scope — instrumented at the adapter boundary. Remaining GT-544/545/547/548/549/551 need infra (Grafana/Prometheus/compose) and stay PENDING. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…core-api (GT-549) /metrics leaks operational shape (routes, request/error volumes, and — since GT-546 — agent-execution counts) and was public on two of three services while core-api guards it (GT-393). Closes that gap: - agent-runtime-api: new fail-closed MetricsAuthGuard (mirrors core-api's) applied to the metrics controller — requires AGENT_RUNTIME_API_KEY (Bearer/x-api-key, constant-time compare) unless AGENT_RUNTIME_ALLOW_NO_AUTH=true (the runtime's own trusted-network escape hatch, so local/in-cluster scraping behind a NetworkPolicy is unaffected). - mcp-server: the raw-HTTP /metrics branch now runs through the same authenticateHttpRequest the MCP endpoints use (shared API key / OAuth / local JWT / allowNoAuth), so an anonymous scrape gets 401 and a credentialled one still succeeds. Verified: mcp-server 326/326 (+2 HTTP integration tests: anon /metrics→401, keyed→200), agent-runtime-api 77/77 (+5 guard tests), both tsc clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes GT-549 (guarded /metrics on agent-runtime + mcp, e99064e). Board 514→515 done · 16→15 pending. Guard 08 green (551 gaps / 497 records). GT-545 (scrape config) dependency dispositioned accepted-scope — the credential-accepts/anon-rejects behaviour is proven by tests; wiring the actual scrape job is the GT-545 infra item. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…eQL, GT-549) CodeQL (js/insufficient-password-hash) flagged the sha256-to-fixed-length step in MetricsAuthGuard.safeEqual. It was a false positive (constant-time compare of a high-entropy shared secret, not password storage), but resolved genuinely by dropping the hash: length-guarded timingSafeEqual over the raw buffers is constant-time for equal-length inputs and unambiguously not password hashing. Guard tests 5/5. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa169b7fbe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… tighten HITL metric, instrument streamed runs - prometheus-alerts.yml + core-api-slo.md/.es.md: the emitted HTTP counter is multi-label (method/path/status), so HighErrorRate is now sum(5xx)/sum(total) (a real percentage, matching the annotation) and the error-budget burn-rate queries aggregate with sum() before dividing; latency quantiles use sum by (le). Without this the vector division mismatched labels and the '> 0.01' compared an absolute rate, not a 1% error rate. - agent-metrics: stop counting every 'blocked' run as a HITL decision — a block can be policy/gate-driven, not a human approval; count ONLY approvals the trace attests (approvedBy). Test updated. - agent-runtime.controller: instrument the SSE /stream path too — the terminal RuntimeEvent carries the assembled result, so streamed executions now emit run metrics at parity with handle()/converse(). Verified: agent-runtime-api 77/77, core-api metrics+drift 10/10, guard 08 green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotes the code-actionable observability wave + the GT-451 CLI packaging chore from develop to main.
Closed gaps (code + tests + guard-08-compliant board)
recordGateEvaluationnow emitsevolith_gate_evaluations_total{status,gateId,phase}+ duration histogram from the evaluate/gate paths.evolith_http_request_duration_secondshistogram (theHighLatencyalert + latency SLOs were inert) and reconciledprometheus-alerts.yml+core-api-slo.md/.es.mdfrom barehttp_requests_totalto the emittedevolith_-prefixed names.AgentMetricsServiceadds the AI-era execution signal (evolith_agent_runs_total{engine,verdict}, run duration, skill invocations, Core calls, HITL approvals) on the default registry.metric-drift.spec.tsfails the PR when an alert/SLO references an Evolith metric no service emits (the root cause of GT-543/545)./metricson agent-runtime + mcp is now guarded (fail-closed with a trusted-network opt-out) for parity with core-api's guarded/metrics.Also included:
62afd9bd(dropdist/.tsbuildinfofrom the CLI tarball, GT-451 prep) and6acb9d13(register the GT-542…551 observability wave), from the concurrent gap-analysis line.Verification
Remaining observability gaps (GT-544/545/547/548/551) need infra (Grafana/Prometheus/compose) and stay PENDING.
🤖 Generated with Claude Code