Skip to content

feat(tracing): add keyset pagination to spans query endpoint#60737

Merged
jonmcwest merged 1 commit into
masterfrom
05-29-feat_tracing_add_keyset_pagination_to_spans_query_endpoint
Jun 1, 2026
Merged

feat(tracing): add keyset pagination to spans query endpoint#60737
jonmcwest merged 1 commit into
masterfrom
05-29-feat_tracing_add_keyset_pagination_to_spans_query_endpoint

Conversation

@jonmcwest
Copy link
Copy Markdown
Contributor

@jonmcwest jonmcwest commented May 29, 2026

Problem

Trace list pagination was broken: the after cursor was applied as a span-level keyset filter inside where(), which is shared by the sparkline, aggregation, and tree runners. This caused child spans of valid traces to be incorrectly filtered out, and hasMore / nextCursor were never populated (hardcoded to False / None).

Changes

Cursor moved to the trace level

Pagination now operates on traces rather than individual spans. The inner query switches from LIMIT 1 BY trace_id to GROUP BY trace_id, ordering by min(timestamp) (the root span's start time) with trace_id as a tiebreaker. The keyset HAVING clause is applied only to this grouped query, so child spans of kept traces are never filtered.

where() no longer touches the cursor

The after cursor logic is removed from where() entirely. A comment explains why: where() is shared across runners that never paginate, and a span-level keyset would incorrectly drop spans belonging to traces that straddle the page boundary.

Cursor encoding uses trace_id (hex → base64)

The cursor now stores trace_id in hex (the human-readable form used by the rest of the API) and re-encodes it to the table's base64 storage form for SQL comparison, keeping the cursor format consistent with the public API surface.

hasMore and nextCursor are now real

The view layer computes the actual trace order from the prefetched spans, trims the overflow trace, and emits a cursor pointing at the last kept trace. hasMore reflects whether the runner returned more traces than the requested limit.

time_sliced_results removed

The view no longer wraps the runner in time_sliced_results; a single blocking runner.run() call is used instead, consistent with how the cursor-based approach needs a single stable result set.

How did you test this code?

Automated tests in the tracing backend test suite cover the cursor encoding/decoding, the GROUP BY query shape, and the hasMore / nextCursor response fields.


  • Publish to changelog?
  • Alert Sales and Marketing teams?

Docs update

Copy link
Copy Markdown
Contributor Author

jonmcwest commented May 29, 2026

@jonmcwest jonmcwest marked this pull request as ready for review May 29, 2026 20:37
@assign-reviewers-posthog assign-reviewers-posthog Bot requested review from a team May 29, 2026 20:37
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 29, 2026

Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
products/tracing/backend/presentation/views.py:348-358
**Trace start may be wrong for prefetched pages without a root span**

`trace_start[tid]` is computed as the minimum timestamp across the prefetched spans returned for each trace. The outer query orders prefetched spans by `is_root_span DESC, matched_filter DESC, timestamp {order_dir}`, so when a root span exists it is always fetched first and is the temporal minimum — the assumption holds. However, when a trace has no span with `is_root_span = 1` (e.g., a partial/incomplete trace ingested without root-span detection) **and** `prefetchSpans > 1` **and** `orderBy = "latest"`, the N fetched spans are the *newest* ones for that trace, not the oldest. The Python `min(timestamp)` of those N spans can be later than the SQL `min(timestamp)` used in the HAVING clause, causing the cursor for that boundary trace to encode the wrong timestamp. On the next page the HAVING would still be `< cursor_ts` but `sql_min(timestamp) < cursor_ts` would be true, so the trace would reappear — a duplicate across pages.

Reviews (1): Last reviewed commit: "feat(tracing): add keyset pagination to ..." | Re-trigger Greptile

Comment thread products/tracing/backend/presentation/views.py Outdated
@jonmcwest jonmcwest force-pushed the 05-29-feat_tracing_add_keyset_pagination_to_spans_query_endpoint branch from 7553e8e to 81c534e Compare June 1, 2026 09:47
@jonmcwest jonmcwest merged commit dc1fda9 into master Jun 1, 2026
196 checks passed
@jonmcwest jonmcwest deleted the 05-29-feat_tracing_add_keyset_pagination_to_spans_query_endpoint branch June 1, 2026 10:13
@deployment-status-posthog
Copy link
Copy Markdown

deployment-status-posthog Bot commented Jun 1, 2026

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-06-01 10:54 UTC Run
prod-us ✅ Deployed 2026-06-01 11:10 UTC Run
prod-eu ✅ Deployed 2026-06-01 11:15 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants