feat: Exemplars for metric & PromQL charts - #2536
Closed
jordan-simonovski wants to merge 24 commits into
Closed
Conversation
🦋 Changeset detectedLatest commit: 178c980 The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
github-actions
Bot
added
the
review/tier-4
Contributor
🔴 Tier 4 — CriticalTouches auth, data models, config, tasks, OTel pipeline, ClickHouse, or CI/CD. Why this tier:
Review process: Deep review from a domain expert. Synchronous walkthrough may be required. Stats
|
jordan-simonovski
marked this pull request as draft
June 29, 2026 05:57
Contributor
Greptile SummaryThe PR adds feature-gated exemplar overlays for eligible metric and PromQL charts, linking chart markers to representative traces.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains; the previously reported endpoint-resolution, metric-scoping, and series-filtering defects are corrected in the current code.
|
| Filename | Overview |
|---|---|
| packages/api/src/config.ts | Guards optional remote write on a configured API-side endpoint so generated collector configuration receives a concrete value. |
| packages/api/src/opamp/controllers/opampController.ts | Adds feature-gated spanmetrics pipelines and embeds the resolved Prometheus remote-write endpoint. |
| packages/common-utils/src/core/renderChartConfig.ts | Renders exemplar SQL while preserving series conditions and independently ANDing the required metric-name predicate. |
| packages/app/src/hooks/useExemplars/useExemplars.tsx | Coordinates feature-gated exemplar retrieval for eligible metric and PromQL chart configurations. |
| packages/app/src/components/DBTimeChart/DBTimeChart.tsx | Integrates sampled exemplar markers and trace metadata into time-chart interaction. |
| docker-compose.dev.yml | Adds local Prometheus exemplar storage and coherent test telemetry generation for development validation. |
Sequence Diagram
sequenceDiagram
participant Span as Instrumented service
participant Collector as OTel Collector
participant Metrics as ClickHouse or Prometheus
participant App as HyperDX chart
participant Trace as Trace source
Span->>Collector: Export trace
Collector->>Collector: Derive span metric with exemplar
Collector->>Metrics: Export metric and trace identifier
App->>Metrics: Query chart series and exemplars
Metrics-->>App: Series plus exemplar points
App->>Trace: Fetch marker trace metadata
Trace-->>App: Service, span, duration, status
Reviews (19): Last reviewed commit: "fix(exemplars): stop a hover card cancel..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
E2E Test Results✅ All tests passed • 266 passed • 1 skipped • 1110s
Tests ran across 4 shards in parallel. |
jordan-simonovski
marked this pull request as ready for review
June 29, 2026 20:58
…ated collector config, so the collector container no longer needs SPAN_METRICS_PROM_RW_ENDPOINT in its own environment.
Three P0/P1 and about sixteen P2 findings from the deep review, plus six more
found by reviewing the fixes themselves. That review ran degraded (no shell, 6
of 9 reviewers, scope read from the working tree rather than a diff), so each
finding was checked against real code first. Two needed a different fix than
suggested; both are called out below.
The P0/P1s:
- The hover card never showed the exemplar's value or time, yet BOTH clamps
justified moving markers on the grounds that it did. It now shows them, above
the trace-source section so a chart with no trace source still gets them.
This was a comment asserting a guarantee the code did not provide, and two
more of the same kind turned up later in this batch.
- placeholderData deliberately keeps the previous range's exemplars across a
range change, and clampExemplarX pinned those out-of-domain markers onto the
new axis edge: real, clickable traces on buckets they never occurred in, with
both isLoading and isError reporting settled. The clamp now nudges within one
bucket (the newest-partial-bucket case it exists for) and drops anything
further. Both clamps moved into useExemplarMarkers so the hover and pin reset
effects key on what is actually drawn.
- Single-series enforcement was inferred from the exemplar payload, which only
contains series that carry a sampled exemplar — so it answered "how many
series had exemplars", not "how many lines are drawn". The count now comes
from the main query. Proving it from the expression instead, as the review
suggested, was tried and rejected: it silently kills
histogram_quantile(0.95, rate(x_bucket[5m])), which has no `by` clause.
The P2s, grouped:
Scale mismatches, where a marker was pinned onto an axis measured in something
else. isExemplarEligible now considers the aggregation, so a count-aggregated
histogram no longer gets duration markers in a count domain. clampExemplarY is
no longer symmetric: pinning down to the ceiling says "at least this high", but
raising a fast request to a fitted floor draws it level with the slowest ones,
so below-floor markers are dropped instead.
Windowing. The scan's time predicate was on the row's TimeUnix while the
projected value is the ARRAY JOINed ex_TimeUnix, so pre-window exemplars came
back and in-window ones whose data point landed later did not. There is now an
exact ex_TimeUnix bound, and the row bound is widened forward by the greater of
the granularity and one minute — the granularity alone resolves to 15s on a
short window, finer than a typical scrape. The bucket index is clamped, so an
inclusive range no longer produces 201 buckets and drops the newest exemplar.
renderMetricExemplarsChartConfig now returns null without a dateRange.
Caching. The query key rounded both range ends to 30s, which collided different
sub-minute windows. It now floors the start and ceils the end, and — the part
the first attempt got wrong — the quantised window is what gets FETCHED, not
just keyed. Keying on it while fetching the raw range left the entry holding
whichever window arrived first.
PromQL parsing. `{code!="200"}` matched the operator test and suppressed the
overlay on a query that does aggregate to one line. `by ("le")` produced a
keep-set matching no real label, collapsing every series into one group. And
stripping comments before strings truncated at a `#` inside a label value,
eating the `by (...)` clause — a fail-closed introduced while fixing the first
two, caught by reviewing the fix.
UI. The Inspect deep link dropped the timestamp, so the search page fell back
to the last 14 days and an older dashboard opened an empty trace view; it now
carries a window. "Compare to Previous Period" put a second line in lineData,
which made the new series-count guard blank the overlay and tell the user to
aggregate to a single line they already had. The pinned card unpins on a range
change, quantised so a live-tail tick does not yank it away a second after the
click.
Tests. Four gating tests asserted synchronously on the first render and passed
with the `enabled` gate hardcoded true; they now flush, assert both fetch
paths, and sit beside a control that proves the harness does fetch. Two tests
written in this batch were themselves vacuous and are fixed or removed. Added
coverage for the placeholder chart-identity guard, an ineligible PromQL
expression, the hover card's value and time, the widening floor, and
resolveExemplarWindow, which was extracted from the route handler so it can be
unit tested without Docker.
Also removed: visibleSeriesMax ran an O(rows x series) pass on every time chart
even with the overlay off and was returned but never read; ifOverflow is now
stated on ReferenceDot instead of relying on the default both clamps work
around; provesSingleSeries, the rejected alternative above, is gone rather than
left as dead code with a docblock reading like the live rule.
Not done: the three per-bucket capping implementations (client, SQL, render)
are still three. Unifying them is a real refactor and the divergence is
documented where it matters.
make ci-lint and make ci-unit pass (5296 tests); dashboard E2E 81 passed with
one unrelated retry. Several fixes were confirmed load-bearing by reverting
them and watching the new test fail.
Contributor
|
<!-- deep-review --> Deep Review🔴 P0/P1 — must fix
🟡 P2 — recommended
🔵 P3 nitpicks (17)
Reviewers (11): correctness, security, adversarial, testing, maintainability, project-standards, reliability, performance, api-contract, kieran-typescript, julik-frontend-races. Testing gaps:
Reviewer coverage caveat: |
One P1 and six P2s. Each was checked against real code before being fixed. The P1 is pre-existing on main (973d120), not from this branch: proxyToPrometheus forwarded the upstream content-type with no nosniff, so a member-configured connection host returning text/html would render as script on our origin — /api/* is same-origin-proxied and the session cookie is sameSite lax. This PR adds a GET route to that proxy, which widens the exposure, so it is fixed here. nosniff always, and the content-type passes through only for a JSON media type. The P2s: - The window split gated on ceil(budget * 0.75) rather than the budget, so 12 populated buckets at the default budget of 12 drew 9 markers and left three bare. - collapsesHistogramBuckets tested a literal `histogram_quantile(` while isPromqlExemplarEligible allows whitespace before the paren. The two disagreed about `histogram_quantile (0.95, ...)`: the toggle allowed it, `le` stayed in the group key, and the overlay came back suppressed telling the user to aggregate to a single line they already had. - The hover card read "Trace not found in source" for three different states: a real miss, a failed query, and a source that is not a Trace kind. The last two now get their own message. - Switching display type swaps AreaChart for BarChart, a different element type, so the whole subtree remounts and every marker unmounts without a mouseleave. An open card was left over markers that no longer exist. Cleared on the switch. - A marker sliding out from under a stationary cursor on a live-tail tick also fires no mouseleave, leaving the card at stale coordinates and the series tooltip suppressed. The range-change effect now clears hover as well as pin. - Markers dropped by the render-layer clamps were invisible to the fetch-layer notice, so the overlay could thin out with nothing on screen explaining why. The count is reported up to the toolbar. Two of these were flagged by my own review last round and filed as residual risks instead of being fixed — the legend-isolation y floor and the missing drop count. They came back as findings, which is the argument against that disposition. On the y floor I took the review's second option (surface the drop) rather than its first (fit the floor only under fitYAxisToData). The first changes y-axis behaviour for every chart in the app, which is wider than this PR should reach. make ci-lint and make ci-unit pass (5300 tests). Dashboard E2E run three times, 0 failed each time; the flake count moved 1/5/3 with different tests each run, all in listing-page specs that draw no chart. The new integration tests for the header hardening are typechecked but not executed — they need Docker.
Contributor
|
<!-- deep-review --> Deep Review🔴 P0/P1 — must fix
🟡 P2 — recommended
🔵 P3 nitpicks (12)
Reviewers (12): correctness, security, adversarial, reliability, performance, testing, maintainability, project-standards, api-contract, typescript, agent-native, collector-infra. Testing gaps:
Coverage note: |
…tted reports The P1 from the latest review: the hover card is a pointer-live absolutely positioned sibling of the chart, so a drag-to-zoom whose path crossed a marker mounted the card under the moving cursor. The cursor entered the card, the chart saw mouseleave and cleared highlightStart and mouseDownPosRef, and the zoom was cancelled halfway through. Exemplar hover is now suppressed while a brush drag is in progress. The alternative — pointer-events: none on the unpinned card — was not taken because it also stops the cursor travelling into the card to press Inspect, which is what the close delay exists for. Also removes packages/common-utils/reports/ from the repo and gitignores it. I committed 46,985 lines of generated Stryker mutation-test output in d93f0f8 by running `git add -A packages` while that untracked directory was present. It is not on main and was never meant to be here; it was most of this PR's apparent size. The ignore is scoped to packages/*/reports/ rather than a bare reports/, which would have hidden real directories elsewhere in the tree. make ci-lint and make ci-unit pass.
Contributor
Author
|
Latest P0/P1 fixed in 178c980 (hover card was cancelling a brush-zoom by mounting under the moving cursor). Also removed Outstanding P2/P3s tracked in #2804 rather than another round here. The overlay is behind |
Contributor
Author
|
Closing this in favour of a stack of smaller PRs. Nothing is lost — the WhyThis PR reached 85 files. It went through four rounds of deep review, each Two packaging errors are worth naming, both mine:
The replacement stackEach targets the one before it where there is a real dependency, and each builds
Outstanding P2s and P3s are tracked in #2804 rather than being re-litigated each I will link each PR here as it goes up. |
This was referenced Aug 5, 2026
Contributor
Deep Review✅ No critical issues found. This is unusually defensive code: both ingest boundaries ( 🟡 P2 -- recommended
🔵 P3 nitpicks (10)
Reviewers (10): correctness, security, testing, maintainability, project-standards, api-contract, reliability, performance, kieran-typescript, agent-native. An adversarial reviewer was also dispatched but did not return before synthesis; its focus areas (unbounded marker counts, feature-flag bypass, stale persisted config, hover/pin/zoom races, malformed upstream shapes) were verified directly against the code instead. Testing gaps:
|
jordan-simonovski
added a commit
that referenced
this pull request
Aug 5, 2026
Time charts on metric and PromQL sources can overlay exemplars — individual trace-linked points — behind NEXT_PUBLIC_ENABLE_EXEMPLARS, off by default, and per-chart behind enableExemplars. Hovering a marker shows the exemplar's own value and time plus trace metadata, with a button to open the trace. The shared common-utils primitives ship here rather than separately: every one of those exports is consumed only by the app, so landing them alone would fail the unused-export check. The rule the whole feature turns on is that a marker sits at the trace's own measurement on the chart's shared axis, so it is only honest when the chart draws one line in the same unit. That is enforced in four places: a single non-ratio histogram series with no group by; an aggregation that leaves the axis on the observation scale (a count of observations is not a duration); for PromQL an expression that plots a duration, with the duration call spanning the whole expression; and a rendered-series count taken from the main query rather than the exemplar response, since Prometheus only returns series that carry a sampled exemplar and so cannot say how many lines are drawn. Where a marker cannot be drawn honestly it is dropped, not moved. Out of the rendered window by more than one bucket, or below a fitted y-axis floor, and it does not render — with the count surfaced on the chart, because a silently thinning overlay is worse than an explained one. The exception is above the ceiling, where pinning reads as "at least this high" and the card carries the real number. Includes the four rounds of review fixes from #2536: the hover card showing value and time (both clamps cite it as their justification), the PromQL aggregation parsing that decides series identity, the ClickHouse scan's ex_TimeUnix bound, the query-key quantisation, and the card lifecycle across zoom, live tail and display-type switches. make ci-lint and make ci-unit pass (5350 tests). Dashboard E2E 80 passed, 0 failed, 2 unrelated flakes on listing-page specs. Stacked on the chart-file refactor so this reads as a feature diff.
jordan-simonovski
added a commit
that referenced
this pull request
Aug 5, 2026
Time charts on metric and PromQL sources can overlay exemplars — individual trace-linked points — behind NEXT_PUBLIC_ENABLE_EXEMPLARS, off by default, and per-chart behind enableExemplars. Hovering a marker shows the exemplar's own value and time plus trace metadata, with a button to open the trace. The shared common-utils primitives ship here rather than separately: every one of those exports is consumed only by the app, so landing them alone would fail the unused-export check. The rule the whole feature turns on is that a marker sits at the trace's own measurement on the chart's shared axis, so it is only honest when the chart draws one line in the same unit. That is enforced in four places: a single non-ratio histogram series with no group by; an aggregation that leaves the axis on the observation scale (a count of observations is not a duration); for PromQL an expression that plots a duration, with the duration call spanning the whole expression; and a rendered-series count taken from the main query rather than the exemplar response, since Prometheus only returns series that carry a sampled exemplar and so cannot say how many lines are drawn. Where a marker cannot be drawn honestly it is dropped, not moved. Out of the rendered window by more than one bucket, or below a fitted y-axis floor, and it does not render — with the count surfaced on the chart, because a silently thinning overlay is worse than an explained one. The exception is above the ceiling, where pinning reads as "at least this high" and the card carries the real number. Includes the four rounds of review fixes from #2536: the hover card showing value and time (both clamps cite it as their justification), the PromQL aggregation parsing that decides series identity, the ClickHouse scan's ex_TimeUnix bound, the query-key quantisation, and the card lifecycle across zoom, live tail and display-type switches. make ci-lint and make ci-unit pass (5350 tests). Dashboard E2E 80 passed, 0 failed, 2 unrelated flakes on listing-page specs. Stacked on the chart-file refactor so this reads as a feature diff.
jordan-simonovski
added a commit
that referenced
this pull request
Aug 6, 2026
HDXMultiSeriesTimeChart.tsx 1503 -> 720 (+ 8 files)
DBTimeChart.tsx 1022 -> 469 (+ 6 files)
Each becomes a directory with a barrel, so every import path and every
jest.mock('@/...') keeps resolving and the change is invisible to consumers.
The seams follow what the code already separated rather than cutting by length:
- searchUrl.ts takes buildSearchUrl out of a useCallback as a pure function,
which makes its branching testable for the first time — which value column a
series key resolves to, and whether that column's aggregation is attributable
to individual events at all. A non-attributable aggregation must not produce a
value filter, or drill-down returns rows that never contributed to the clicked
point. Nine new tests cover it; that is the only new test surface here.
- useChartScales holds the axis domains and the annotation elements. Pure
derivation from props, no state, no recharts tree.
- The tooltip, legend, recharts shape shims, layout constants, cross-chart pin
registry and pure data helpers each move to a file named after what they are.
MemoChart stays at 720 because what is left is one recharts element whose
children must remain siblings, plus interaction state that genuinely shares a
click-suppression flag with the brush-zoom. Splitting that tree would trade a
real risk of dropping a render branch for a smaller number.
Two deliberate changes beyond pure movement, both small:
- xAxisDomain is typed as the [number, number] tuple it already returned rather
than the wider AxisDomain, which removes two unsafe type assertions at its
consumers.
- dismissPinned and buildSearchUrl became block-bodied and a thin delegate
respectively. Same behaviour.
Verified by diffing every non-import line of the two originals against the new
directories: the only lines that do not appear are import fragments that were
redistributed and the two changes above.
make ci-lint, make ci-unit (5205 tests) and the dashboard E2E suite (81 passed)
all pass. E2E matters most here — it is the only check that catches a
runtime-only breakage from moving code.
First of the pieces split out of #2536. No feature code: this lands ahead of the
exemplars overlay so that PR is a feature diff rather than a feature plus a
1,500-line move.
jordan-simonovski
added a commit
that referenced
this pull request
Aug 6, 2026
HDXMultiSeriesTimeChart.tsx 1503 -> 720 (+ 8 files)
DBTimeChart.tsx 1022 -> 469 (+ 6 files)
Each becomes a directory with a barrel, so every import path and every
jest.mock('@/...') keeps resolving and the change is invisible to consumers.
The seams follow what the code already separated rather than cutting by length:
- searchUrl.ts takes buildSearchUrl out of a useCallback as a pure function,
which makes its branching testable for the first time — which value column a
series key resolves to, and whether that column's aggregation is attributable
to individual events at all. A non-attributable aggregation must not produce a
value filter, or drill-down returns rows that never contributed to the clicked
point. Nine new tests cover it; that is the only new test surface here.
- useChartScales holds the axis domains and the annotation elements. Pure
derivation from props, no state, no recharts tree.
- The tooltip, legend, recharts shape shims, layout constants, cross-chart pin
registry and pure data helpers each move to a file named after what they are.
MemoChart stays at 720 because what is left is one recharts element whose
children must remain siblings, plus interaction state that genuinely shares a
click-suppression flag with the brush-zoom. Splitting that tree would trade a
real risk of dropping a render branch for a smaller number.
Two deliberate changes beyond pure movement, both small:
- xAxisDomain is typed as the [number, number] tuple it already returned rather
than the wider AxisDomain, which removes two unsafe type assertions at its
consumers.
- dismissPinned and buildSearchUrl became block-bodied and a thin delegate
respectively. Same behaviour.
Verified by diffing every non-import line of the two originals against the new
directories: the only lines that do not appear are import fragments that were
redistributed and the two changes above.
make ci-lint, make ci-unit (5205 tests) and the dashboard E2E suite (81 passed)
all pass. E2E matters most here — it is the only check that catches a
runtime-only breakage from moving code.
First of the pieces split out of #2536. No feature code: this lands ahead of the
exemplars overlay so that PR is a feature diff rather than a feature plus a
1,500-line move.
jordan-simonovski
added a commit
that referenced
this pull request
Aug 6, 2026
Time charts on metric and PromQL sources can overlay exemplars — individual trace-linked points — behind NEXT_PUBLIC_ENABLE_EXEMPLARS, off by default, and per-chart behind enableExemplars. Hovering a marker shows the exemplar's own value and time plus trace metadata, with a button to open the trace. The shared common-utils primitives ship here rather than separately: every one of those exports is consumed only by the app, so landing them alone would fail the unused-export check. The rule the whole feature turns on is that a marker sits at the trace's own measurement on the chart's shared axis, so it is only honest when the chart draws one line in the same unit. That is enforced in four places: a single non-ratio histogram series with no group by; an aggregation that leaves the axis on the observation scale (a count of observations is not a duration); for PromQL an expression that plots a duration, with the duration call spanning the whole expression; and a rendered-series count taken from the main query rather than the exemplar response, since Prometheus only returns series that carry a sampled exemplar and so cannot say how many lines are drawn. Where a marker cannot be drawn honestly it is dropped, not moved. Out of the rendered window by more than one bucket, or below a fitted y-axis floor, and it does not render — with the count surfaced on the chart, because a silently thinning overlay is worse than an explained one. The exception is above the ceiling, where pinning reads as "at least this high" and the card carries the real number. Includes the four rounds of review fixes from #2536: the hover card showing value and time (both clamps cite it as their justification), the PromQL aggregation parsing that decides series identity, the ClickHouse scan's ex_TimeUnix bound, the query-key quantisation, and the card lifecycle across zoom, live tail and display-type switches. make ci-lint and make ci-unit pass (5350 tests). Dashboard E2E 80 passed, 0 failed, 2 unrelated flakes on listing-page specs. Stacked on the chart-file refactor so this reads as a feature diff.
jordan-simonovski
added a commit
that referenced
this pull request
Aug 6, 2026
HDXMultiSeriesTimeChart.tsx 1503 -> 720 (+ 8 files)
DBTimeChart.tsx 1022 -> 469 (+ 6 files)
Each becomes a directory with a barrel, so every import path and every
jest.mock('@/...') keeps resolving and the change is invisible to consumers.
The seams follow what the code already separated rather than cutting by length:
- searchUrl.ts takes buildSearchUrl out of a useCallback as a pure function,
which makes its branching testable for the first time — which value column a
series key resolves to, and whether that column's aggregation is attributable
to individual events at all. A non-attributable aggregation must not produce a
value filter, or drill-down returns rows that never contributed to the clicked
point. Nine new tests cover it; that is the only new test surface here.
- useChartScales holds the axis domains and the annotation elements. Pure
derivation from props, no state, no recharts tree.
- The tooltip, legend, recharts shape shims, layout constants, cross-chart pin
registry and pure data helpers each move to a file named after what they are.
MemoChart stays at 720 because what is left is one recharts element whose
children must remain siblings, plus interaction state that genuinely shares a
click-suppression flag with the brush-zoom. Splitting that tree would trade a
real risk of dropping a render branch for a smaller number.
Two deliberate changes beyond pure movement, both small:
- xAxisDomain is typed as the [number, number] tuple it already returned rather
than the wider AxisDomain, which removes two unsafe type assertions at its
consumers.
- dismissPinned and buildSearchUrl became block-bodied and a thin delegate
respectively. Same behaviour.
Verified by diffing every non-import line of the two originals against the new
directories: the only lines that do not appear are import fragments that were
redistributed and the two changes above.
make ci-lint, make ci-unit (5205 tests) and the dashboard E2E suite (81 passed)
all pass. E2E matters most here — it is the only check that catches a
runtime-only breakage from moving code.
First of the pieces split out of #2536. No feature code: this lands ahead of the
exemplars overlay so that PR is a feature diff rather than a feature plus a
1,500-line move.
jordan-simonovski
added a commit
that referenced
this pull request
Aug 6, 2026
Time charts on metric and PromQL sources can overlay exemplars — individual trace-linked points — behind NEXT_PUBLIC_ENABLE_EXEMPLARS, off by default, and per-chart behind enableExemplars. Hovering a marker shows the exemplar's own value and time plus trace metadata, with a button to open the trace. The shared common-utils primitives ship here rather than separately: every one of those exports is consumed only by the app, so landing them alone would fail the unused-export check. The rule the whole feature turns on is that a marker sits at the trace's own measurement on the chart's shared axis, so it is only honest when the chart draws one line in the same unit. That is enforced in four places: a single non-ratio histogram series with no group by; an aggregation that leaves the axis on the observation scale (a count of observations is not a duration); for PromQL an expression that plots a duration, with the duration call spanning the whole expression; and a rendered-series count taken from the main query rather than the exemplar response, since Prometheus only returns series that carry a sampled exemplar and so cannot say how many lines are drawn. Where a marker cannot be drawn honestly it is dropped, not moved. Out of the rendered window by more than one bucket, or below a fitted y-axis floor, and it does not render — with the count surfaced on the chart, because a silently thinning overlay is worse than an explained one. The exception is above the ceiling, where pinning reads as "at least this high" and the card carries the real number. Includes the four rounds of review fixes from #2536: the hover card showing value and time (both clamps cite it as their justification), the PromQL aggregation parsing that decides series identity, the ClickHouse scan's ex_TimeUnix bound, the query-key quantisation, and the card lifecycle across zoom, live tail and display-type switches. make ci-lint and make ci-unit pass (5350 tests). Dashboard E2E 80 passed, 0 failed, 2 unrelated flakes on listing-page specs. Stacked on the chart-file refactor so this reads as a feature diff.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Engineers staring at a latency spike on a chart have no way to jump to a trace that caused it.
This change adds exemplars, clickable markers overlaid on time charts, each linking to a representative trace.
Works for metric and PromQL sources.
Also added some test-telemetry infrastructure to build and validate the feature against more complex data sets.
CleanShot.2026-06-29.at.13.50.52.mp4
Exemplar overlay (app)
Two data backends
Fully-OTLP coherent metrics (collector)
Telemetry generator (telemetry-generator/)
Team setting
Scoping
Out of scope (separate tickets)
Testing
Changesets
Notes / caveats
Storybook: exemplar components
Extracted the two exemplar UI pieces — the
ExemplarDotchart marker and theExemplarHoverCardtrace popover — out ofHDXMultiSeriesTimeChartandDBTimeChartinto a focusedcomponents/Exemplars/directory, with a Storybook story for each. The card story covers every state (full/partial metadata, loading, trace-not-found, no-trace-source-configured) and the dot story renders the marker in isolation, so both can be reviewed across light/dark and both brand themes without a live ClickHouse query. Behaviour-preserving;ExemplarDotalso gains a real props type in place ofany.Merged
mainto bring the branch current — this includes the recharts 2→3 upgrade, for which the exemplarReferenceDotmarker was adjusted to the v3 API.Review hardening & feature gate (latest)
Follow-up addressing review feedback (Greptile P1 + Deep Review P2s) and gating the feature for a safe rollout:
NEXT_PUBLIC_ENABLE_EXEMPLARS: off by default, enabled in local dev. It can ship dark while we finish testing against real data.filtersLogicalOperator: 'OR'can no longer let the exemplar scan match other metrics.enableExemplarsis cleared when a chart leaves single-series so a stale flag can't linger.Note: the exemplar marker value was verified correct — it matches the span's real duration to the nanosecond; the earlier "off by an order of magnitude" appearance was a single slow outlier stretching the y-axis, addressed above.