experiments: promoted-vs-original traffic over time - #70
Conversation
Add a "Promoted vs original — median goodput" scatter to the Experiments tab. Each point is a promoted track (an experiment's challenger) plotted against the original it beat (the control), measured over a recent window in the experiment's target market. A dashed parity line splits winning (above) from losing (below) promotions, so regressions stand out. Filters: country / protocol / provider dropdowns plus a time-range selector (6h/24h/7d/30d). A point is plotted only when both arms have live samples in the window; the rest are counted as hidden rather than piled on the origin. Backed by the new /v1/dashboard/experiments/promoted-comparison endpoint (one request, server-side SigNoz query).
|
Warning Review limit reached
Next review available in: 46 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughChangesThe experiments API now exposes promoted-versus-original comparison data. A hook fetches this data for selected time windows, and the Experiments view renders filtered goodput scatter comparisons with loading, error, empty, tooltip, and summary states. Promoted comparison
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant ExperimentsOverview
participant usePromotedComparison
participant fetchPromotedComparison
participant ExperimentsPromotedComparisonEndpoint
ExperimentsOverview->>usePromotedComparison: enable comparison with selected hours
usePromotedComparison->>fetchPromotedComparison: request comparison data
fetchPromotedComparison->>ExperimentsPromotedComparisonEndpoint: GET hours query parameter
ExperimentsPromotedComparisonEndpoint-->>fetchPromotedComparison: return comparison response
fetchPromotedComparison-->>usePromotedComparison: resolve response
usePromotedComparison-->>ExperimentsOverview: provide data, loading, and error state
ExperimentsOverview->>ExperimentsOverview: filter points and render scatter chart
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Adds a new “Promoted vs original — median goodput” scatter visualization to the Experiments tab, backed by a single server-side query via a new dashboard endpoint, to quickly spot post-promotion regressions.
Changes:
- Added
GET /v1/dashboard/experiments/promoted-comparisonclient types + fetch helper and a React hook to load the comparison window. - Implemented a new Recharts-based scatter (parity line + win/loss coloring) with country/protocol/provider filters and selectable time windows.
- Rendered the scatter between the lifecycle pipeline strip and the experiments table.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
src/api/client.ts |
Adds PromotedComparison* types and fetchPromotedComparison(hours) API call. |
src/hooks/useExperiments.ts |
Adds usePromotedComparison(enabled, hours) hook to load scatter data. |
src/components/ExperimentsOverview.tsx |
Adds the PromotedComparison UI (filters, windows, scatter chart) and mounts it in the Experiments view. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/components/ExperimentsOverview.tsx (1)
623-625: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winTime-window chips and Clear are not keyboard-accessible.
These are the only way to change the window, but as bare
<div onClick>they can't be focused or activated via keyboard, and expose no role to assistive tech. Consider<button>(or addrole="button",tabIndex={0}, andonKeyDownfor Enter/Space). The native<select>filters below are already accessible.Also applies to: 651-653
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/components/ExperimentsOverview.tsx` around lines 623 - 625, Update the comparison-window chips in the COMPARISON_WINDOWS map and the Clear control to use keyboard-accessible button elements, preserving their existing click handlers and styling while exposing appropriate button semantics and supporting Enter/Space activation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/components/ExperimentsOverview.tsx`:
- Line 602: Update the hidden count calculation in ExperimentsOverview to derive
from the already filtered points rather than allPoints, so it reflects only
country, protocol, and provider selections. Preserve the existing
promotedSamples and originalSamples exclusion criteria and the filtered track
summary behavior.
---
Nitpick comments:
In `@src/components/ExperimentsOverview.tsx`:
- Around line 623-625: Update the comparison-window chips in the
COMPARISON_WINDOWS map and the Clear control to use keyboard-accessible button
elements, preserving their existing click handlers and styling while exposing
appropriate button semantics and supporting Enter/Space activation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 24461598-1a9c-4e57-842b-c0effc4b2a3f
📒 Files selected for processing (3)
src/api/client.tssrc/components/ExperimentsOverview.tsxsrc/hooks/useExperiments.ts
…ed axis max, chips as accessible buttons
Deploying lantern-dashboard with
|
| Latest commit: |
367179c
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://84ca91ab.lantern-dashboard.pages.dev |
| Branch Preview URL: | https://reflog-promoted-comparison-g.lantern-dashboard.pages.dev |
… filters Turn the pipeline-strip stage cards into toggle buttons that show/hide that status in the experiments list (accessible <button>s with aria-pressed; off = dimmed + struck through). All stages on by default except 'retired', which is the bulk of terminal history. Also address CodeRabbit: the promoted-comparison "N hidden" count now derives from points matching the active country/protocol/provider filters (shared matchesFilters predicate), so it stays consistent with the filtered track count instead of showing a global total.
reflog
changed the title
| <div style={{ ...sectionLabel, marginBottom: "0.15rem" }}>Promoted vs original — median goodput</div> | ||
| <div style={{ ...mono, fontSize: "0.55rem", color: "var(--text-muted)" }}> | ||
| Each point is a promoted track vs the original it beat, over the last {COMPARISON_WINDOWS.find((w) => w.hours === hours)?.label ?? `${hours}h`}, in its target market. Above the parity line = promotion still winning. | ||
| </div> |
| const delta = p.originalGoodput > 0 ? ((p.promotedGoodput - p.originalGoodput) / p.originalGoodput) * 100 : 0; | ||
| const deltaColor = delta >= 0 ? WIN_COLOR : LOSS_COLOR; | ||
| return ( | ||
| <div style={{ ...mono, fontSize: "0.62rem", background: "var(--bg-secondary)", border: "1px solid #ffffff14", borderRadius: "var(--radius-sm)", padding: "0.5rem 0.6rem", lineHeight: 1.5 }}> | ||
| <div style={{ color: "var(--text-muted)" }}>#{p.experimentId} · {p.targetCountry} · {p.protocolName || "—"}{p.providerName ? ` · ${p.providerName}` : ""}</div> | ||
| <div><span style={{ color: WIN_COLOR }}>{p.promotedTrackName}</span> <span style={{ color: "var(--text-muted)" }}>(promoted)</span></div> | ||
| <div><span style={{ color: CONTROL_COLOR }}>{p.originalTrackName}</span> <span style={{ color: "var(--text-muted)" }}>(original)</span></div> | ||
| <div style={{ marginTop: "0.25rem" }}>promoted: {formatBytesPerSec(p.promotedGoodput)} <span style={{ color: "var(--text-muted)" }}>({p.promotedSamples} sess)</span></div> | ||
| <div>original: {formatBytesPerSec(p.originalGoodput)} <span style={{ color: "var(--text-muted)" }}>({p.originalSamples} sess)</span></div> | ||
| <div style={{ color: deltaColor, marginTop: "0.15rem" }}>{delta >= 0 ? "+" : ""}{delta.toFixed(0)}% vs original</div> |
What
Adds a "Promoted vs original — traffic over time" section to the Experiments tab: one small-multiple card per promotion, showing the promoted track vs its control (original) as
proxy.iothroughput over time. Lets you watch the promoted line climb (and the control's market share fall) — i.e. is the promotion actually working?Key design points
geo.country.iso_code = targetCountry). A control is frequently a multi-market incumbent whose total traffic dwarfs a single-market challenger (e.g.hysteria2-oci-free-vpsdoes ~5 MB/s globally but ~450 KB/s in MM) — only the target-market slice is a fair comparison. Oneproxy.ioquery per distinct market, keyed by(track, market).Also in this PR (from earlier review + a screenshot request):
retired).<button>s; misc Copilot/CodeRabbit fixes.Depends on
Test plan
tsc -b✅ ·eslint(changed files) ✅ ·vite build✅proxy.io/proxy.track/transmit, market-scoped) validated against live SigNoz for a real promoted pair.