feat(server): expose stage router stats - #344
Conversation
Signed-off-by: nachiketb <nachiketb@nvidia.com>
|
Signed-off-by: nachiketb <nachiketb@nvidia.com>
Signed-off-by: nachiketb <nachiketb@nvidia.com>
Signed-off-by: nachiketb <nachiketb@nvidia.com>
Signed-off-by: nachiketb <nachiketb@nvidia.com>
Signed-off-by: nachiketb <nachiketb@nvidia.com>
Signed-off-by: nachiketb <nachiketb@nvidia.com>
WalkthroughChangesThe server now collects stage-router counters and histograms through the shared Prometheus registry. Algorithm statistics
Estimated code review effort: 4 (Complex) | ~45 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
crates/switchyard-server/src/stats/algorithms.rs (1)
62-64: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument the crate-visible bucket registry.
Add concise
///documentation forhistogram_buckets. State that it returns explicit histogram boundaries for supported algorithm instruments. As per coding guidelines, add concise///documentation for public Rust items.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/switchyard-server/src/stats/algorithms.rs` around lines 62 - 64, Add concise Rust doc comments to the crate-visible histogram_buckets function, stating that it returns explicit histogram boundaries for supported algorithm instruments. Keep the existing delegation to stage_router::histogram_buckets unchanged.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@crates/switchyard-server/src/stats/algorithms.rs`:
- Around line 62-64: Add concise Rust doc comments to the crate-visible
histogram_buckets function, stating that it returns explicit histogram
boundaries for supported algorithm instruments. Keep the existing delegation to
stage_router::histogram_buckets unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: f755d8a6-976b-4264-807e-23788a1006ec
📒 Files selected for processing (9)
crates/switchyard-server/README.mdcrates/switchyard-server/src/lib.rscrates/switchyard-server/src/metrics.rscrates/switchyard-server/src/stats.rscrates/switchyard-server/src/stats/accumulator.rscrates/switchyard-server/src/stats/algorithms.rscrates/switchyard-server/src/stats/algorithms/stage_router.rscrates/switchyard-server/tests/server.rsdocs/routing_algorithms/stage_router_routing.md
|
can you add a JSON snapshot of how the stats look like before and after? |
Signed-off-by: nachiketb <nachiketb@nvidia.com>
Nice! I dont think sum makes sense as much as the p50, p99 data points. can you remove sum and keep p50 and p99 along with mean + count? |
ayushag-nv
left a comment
ayushag-nv
left a comment
There was a problem hiding this comment.
Internal review come out clean
Signed-off-by: nachiketb <nachiketb@nvidia.com>
3 participants
What
algorithm_stats.stage_routertoGET /v1/statsPOST /v1/stats/resetWhy
The stage router already emits standard OpenTelemetry counters and histograms, but the server did not expose a readable stage-router summary comparable to the deprecated Python server. This makes those values available without adding a custom metrics API to libsy or changing the canonical
/metricsoutput.How
/v1/stats/resetdoes not reset process-lifetime OpenTelemetry metricsFuture algorithm projections can be added under
stats/algorithms/without changing the accumulator, server, Algorithm, or Driver APIs.JSON Shape
Before this MR,
/v1/statsdid not include analgorithm_statsfield.After stage-router traffic, the new fragment looks like this (existing top-level stats fields omitted):
{ "algorithm_stats": { "stage_router": { "routing_decisions": { "dimensions": { "total": 3, "targets": { "model/capable": 1, "model/efficient": 2 } } }, "scoring": { "score": { "count": 2, "mean": 0.125 }, "confidence": { "count": 1, "mean": 0.75 }, "dimensions": { "severity": { "count": 0, "mean": 0.0 }, "spinning": { "count": 0, "mean": 0.0 }, "exploring": { "count": 0, "mean": 0.0 }, "production_intensity": { "count": 0, "mean": 0.0 } } } } } }After
POST /v1/stats/reset, process-lifetime OpenTelemetry metrics remain intact while the JSON projection starts from a new baseline:{ "algorithm_stats": { "stage_router": { "routing_decisions": {}, "scoring": { "score": { "count": 0, "mean": 0.0 }, "confidence": { "count": 0, "mean": 0.0 }, "dimensions": { "severity": { "count": 0, "mean": 0.0 }, "spinning": { "count": 0, "mean": 0.0 }, "exploring": { "count": 0, "mean": 0.0 }, "production_intensity": { "count": 0, "mean": 0.0 } } } } } }What To Review
algorithm_stats.stage_routerJSON shapedecision_sourceand semantictarget_name/v1/stats/resetValidation
cargo test -p switchyard-libsy algorithms::util::stage::testscargo test -p switchyard-server stage_router_projection_preserves_decisions_scores_and_reset_baselinecargo clippy -p switchyard-libsy -p switchyard-server --all-targets -- -D warningscargo fmt --all --checkgit diff --checkNo Python or live-provider tests were needed; this is isolated to Rust algorithm telemetry and server stats projection.