Skip to content

feat(observability): instrument REES + its main-app client with real Prometheus metrics#5398

Merged
JSONbored merged 2 commits into
mainfrom
feat/rees-metrics-instrumentation
Jul 12, 2026
Merged

feat(observability): instrument REES + its main-app client with real Prometheus metrics#5398
JSONbored merged 2 commits into
mainfrom
feat/rees-metrics-instrumentation

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • REES (review-enrichment/) previously had zero /metrics (confirmed live: GET /metrics -> 404) — analyzer success/failure/timeout rates and request outcomes were completely invisible.
  • Adds a minimal, self-contained in-process Prometheus registry to REES (review-enrichment/src/metrics.ts), mirroring the shape of the main app's src/selfhost/metrics.ts pattern but scoped to REES's own much smaller needs (no gauges/redaction — REES is a separate deployable, sometimes run standalone on Railway per its own package description).
  • Instruments brief.ts's runAnalyzer at every one of its 5 outcome branches (ok/degraded/timeout/capped/skipped) with rees_analyzer_runs_total{analyzer,status} (counter) and rees_analyzer_duration_seconds{analyzer} (histogram, only for analyzers that actually executed).
  • Adds GET /metrics to review-enrichment/src/server.ts, and instruments the /v1/enrich route with rees_enrich_requests_total{status} + rees_enrich_request_duration_seconds.
  • Separately instruments the main app's own client-side call in src/review/enrichment-wire.ts::buildReviewEnrichment — this measures REES from the engine's observable perspective (ok/empty/http_error/timeout/exception/skipped_auth_rejected), which is a genuinely different signal than REES's own server-side counters (e.g. it can distinguish a real timeout from the auth-rejected circuit breaker skipping the call entirely). New metric names registered in src/selfhost/metrics.ts's DEFAULT_METRIC_META.
  • Full branch coverage on the src/** changes (Codecov-gated); review-enrichment/ has its own dedicated test suite (node --test, not Codecov-tracked — confirmed via codecov.yml's ignore list only covering src/**).

The Prometheus scrape target and Grafana dashboard — the remaining deliverables from the tracking issue for this work — follow in a separate, smaller PR once this instrumentation is in place to query against.

Part of #5365.

Test plan

  • npm run typecheck
  • npm run test:ci (full local gate: 15046 passed / 12 skipped)
  • npm audit --audit-level=moderate — 0 vulnerabilities
  • review-enrichment's own test suite (npm test inside review-enrichment/): 1330 passed / 0 failed, including 18 new tests for the new metrics module + buildBrief's per-analyzer outcome recording
  • New test/unit/enrichment-wire.test.ts cases cover every new branch (ok/empty/http_error/timeout/exception/skipped_auth_rejected), verified via renderMetrics() output

…Prometheus metrics

REES (review-enrichment/) previously had zero /metrics — analyzer success/failure/
timeout rates and request outcomes were invisible. Adds a minimal in-process
Prometheus registry to REES itself (request outcomes + per-analyzer run/duration,
keyed by analyzer name and status), and separately instruments the main app's
client-side call in enrichment-wire.ts (request outcome + latency, from the
engine's own observable perspective). The Prometheus scrape target and Grafana
dashboard (the remaining deliverables from issue 5367) follow in a separate PR.

Part of #5365
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.40%. Comparing base (1da48c9) to head (d813b39).
⚠️ Report is 5 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5398      +/-   ##
==========================================
- Coverage   94.44%   94.40%   -0.04%     
==========================================
  Files         551      551              
  Lines       44232    44244      +12     
  Branches    14657    14659       +2     
==========================================
- Hits        41773    41768       -5     
- Misses       1784     1801      +17     
  Partials      675      675              
Flag Coverage Δ
shard-1 43.83% <46.15%> (-0.36%) ⬇️
shard-2 34.08% <15.38%> (-0.51%) ⬇️
shard-3 32.18% <15.38%> (+0.64%) ⬆️
shard-4 31.30% <15.38%> (+0.01%) ⬆️
shard-5 33.17% <100.00%> (-0.15%) ⬇️
shard-6 43.58% <15.38%> (+0.20%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/review/enrichment-wire.ts 98.45% <100.00%> (+0.10%) ⬆️
src/selfhost/metrics.ts 100.00% <ø> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 12, 2026
@loopover-orb

loopover-orb Bot commented Jul 12, 2026

Copy link
Copy Markdown

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-12 16:33:35 UTC

8 files · 1 AI reviewer · 2 blockers · readiness 75/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
This PR adds a minimal in-process Prometheus registry to REES and wires it into both REES's own analyzer/request outcomes and the main app's client-side view of REES calls (a genuinely distinct signal set: ok/empty/http_error/timeout/exception/skipped_auth_rejected vs REES's own ok/degraded/timeout/capped/skipped). The implementation is careful and traced correctly: `recordAnalyzerOutcome` omits `elapsedMs` for skip/cap paths that never invoke the analyzer, the auth-rejected circuit breaker skip correctly records a request outcome without a duration sample, and `timeoutMs <= 0` / `shouldStartAnalyzer` capped paths correctly call `releaseAnalyzerCircuitProbe` alongside the new metric call (no regression to the existing #2541 circuit logic). Test coverage is genuinely end-to-end (fetch mocked, real `buildReviewEnrichment`/`buildBrief` invoked, metrics asserted via the real registry), not fabricated stubs. Part of #5365 satisfies the issue-link requirement.

Blockers

  • review-enrichment/src/server.ts:56 records Hono route exceptions only through `app.onError` and has no access to `startedAtMs`, so any thrown `/v1/enrich` failure returns 500 without incrementing `rees_enrich_requests_total{status="error"}` or observing duration; wrap the `/v1/enrich` body in `try/catch` and call `recordEnrichOutcome("error", startedAtMs)` before rethrowing or returning the 500 response.
Nits — 5 non-blocking
  • review-enrichment/src/server.ts:77 hardcodes the 503 status inline right next to the new metrics call; a named constant (or reuse of an existing one if present elsewhere) would make the mapping from status code to `service_not_configured` clearer.
  • review-enrichment/src/brief.ts and src/review/enrichment-wire.ts both cross the 400-line file-size threshold with this change; not a blocker but worth a follow-up split (e.g. extracting the metrics-recording helpers into their own module) if either file grows further.
  • review-enrichment/src/metrics.ts's `renderMetrics` nests to depth 5 in the histogram bucket loop; consider extracting a small `renderHistogram(h)` helper to flatten it, matching the counters loop's simplicity.
  • The REES-side `rees_enrich_requests_total` HELP text (metrics.ts:24) lists status values `ok/unauthorized/service_not_configured/bad_request/error` but the actual code only ever records `ok/unauthorized/service_not_configured/bad_request` (server.ts) — `error` is never emitted since `app.onError` doesn't call `recordEnrichOutcome`; worth fixing the doc string or wiring the error-handler to record it too.
  • `recordReesEnrichOutcome`'s duration-omission logic (enrichment-wire.ts:16) is a good design but relies on callers remembering to pass `startedAtMs` correctly; a type that made the two call shapes (with/without duration) more explicit would reduce future call-site mistakes, though current call sites are all correct.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ⚠️ 1 scoped overlap Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 44 registered-repo PR(s), 36 merged, 463 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 44 PR(s), 463 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 44 PR(s), 463 issue(s).
  • Related work: Titles/paths share 7 meaningful terms. (issue #5367, issue #5368)
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Review top overlaps.
  • Add a concise scope and risk note.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
  • Check active issues and PRs before submitting.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 12, 2026
…"} outcome on a thrown /v1/enrich exception

The /v1/enrich handler recorded ok/service_not_configured/unauthorized/bad_request
outcomes, but a thrown exception (e.g. from buildBrief) skipped straight to Hono's
app.onError, which has no access to startedAtMs -- so a real server-side failure
was invisible in rees_enrich_requests_total despite "error" already being listed
in the metric's own HELP text. Wraps the handler body in try/catch, records the
outcome there, then rethrows so onError's existing 500 response + Sentry capture
are unchanged.
@JSONbored
JSONbored merged commit 15f191a into main Jul 12, 2026
18 checks passed
@JSONbored
JSONbored deleted the feat/rees-metrics-instrumentation branch July 12, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant