fix(webapp): use composite keyset cursor for run pagination

[matt-aitken]

Problem

ClickHouseRunsRepository.listRunIds / listRuns order results by the composite key (created_at, run_id), but the cursor predicate cut on run_id alone:

.where("run_id < {runId: String}", { runId: cursor })
.orderBy("created_at DESC, run_id DESC")

This is only sound when run_id lexicographic order matches created_at order. run_ids are cuids — only coarsely time-sortable — so when a burst of runs is created within a sub-second window, the two orders can diverge. When they do, the next-page predicate (run_id < cursor, where cursor is the last page element = the smallest created_at, not necessarily the smallest run_id):

• re-includes rows already returned on a previous page (duplicates), and
• skips rows it should have returned (silent data loss).

For bulk replay this caused runs to be replayed more than once (replay has no idempotency guard). For the dashboard and the runs.list API it could silently repeat or skip runs at page boundaries.

Fix

Make the cursor predicate match the composite ordering:

• Cursors now encode the full (created_at, run_id) key as an opaque URL-safe base64 token (base64url({"c":<createdAtMs>,"r":"<runId>"})), and the query cuts on the matching tuple — (created_at, run_id) < (…) forward / > (…) backward.
• The ORDER BY is unchanged, so the query stays aligned with the table's primary key — no performance regression (the tuple range predicate is actually more index-friendly than run_id < alone).
• Cursors are server-issued opaque tokens (the SDK only echoes pagination.next / pagination.previous back), so this needs no client/SDK update. Legacy cursors were the bare internal run_id; they're detected by decode failure (a cuid isn't a valid base64-wrapped JSON payload) and fall back to the old run_id-only predicate, so in-flight cursors keep working and drain naturally. New cursors also no longer expose a bare internal run id.
• listRunIds is now the single cursor-aware list primitive: it returns { runIds, pagination: { nextCursor, previousCursor } }, and listRuns builds on it (one place constructs cursors). Bulk actions consume the same method and advance by pagination.nextCursor, finishing when it's null.
• getTaskRunsQueryBuilder now also selects toUnixTimestamp64Milli(created_at) AS created_at_ms, using a dedicated TaskRunListQueryResult schema. The shared TaskRunV2QueryResult stays run_id-only so the run-engine pending-version lookup (getPendingVersionIdsQueryBuilder, which selects only run_id) doesn't fail validation on a column it doesn't query.

Tests

New runsRepositoryCursor.test.ts (testcontainer-backed, real Postgres→ClickHouse replication):

• forward pagination returns every run exactly once when run_id order is the reverse of created_at order (reproduces the duplicate/skip bug — fails on main; this walk-until-nextCursor-null-and-assert-complete is exactly the bulk action's iteration),
• backward pagination round-trips to the previous page across a boundary,
• legacy bare-run_id cursor still uses the old predicate (backwards compatibility).

The existing runsRepository suites (part1–4) still pass; part4's count new runs with listRunIds test was updated for the new { runIds, pagination } return shape, and the clickhouse taskRuns query-builder snapshots were regenerated for the added created_at_ms column.

Notes

• Separate, pre-existing issue (out of scope, not introduced here): listRuns' backward display-slicing (rows.slice(1, size+1) when hasMore) has an off-by-one that can return a straddled page. Tracked separately.

🤖 Generated with Claude Code

[changeset-bot]

⚠️ No Changeset found

Latest commit: 7116971

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

This PR fixes a correctness bug in keyset pagination for ClickHouseRunsRepository where cursor predicates diverged from the query's composite (created_at, run_id) ordering. The fix introduces a new v2 cursor format encoding the composite key, updates ClickHouse query results to include created_at_ms, refactors repository methods to apply matching composite predicates, and adds backwards compatibility for legacy run_id-only cursors. BulkActionService is updated to consume the new cursor pagination API, and comprehensive integration tests verify forward/backward pagination consistency and legacy cursor handling.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'fix(webapp): use composite keyset cursor for run pagination' is specific and directly summarizes the main change: fixing pagination by switching from single-key to composite-key cursor predicates.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	PR description is comprehensive and well-structured with clear Problem, Fix, Tests, and Notes sections explaining the pagination bug, solution, testing approach, and known limitations.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/runs-cursor-keyset-pagination

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[pkg-pr-new]

Open in StackBlitz

@trigger.dev/build

npm i https://pkg.pr.new/@trigger.dev/build@5ee6389

trigger.dev

npm i https://pkg.pr.new/trigger.dev@5ee6389

@trigger.dev/core

npm i https://pkg.pr.new/@trigger.dev/core@5ee6389

@trigger.dev/plugins

npm i https://pkg.pr.new/@trigger.dev/plugins@5ee6389

@trigger.dev/python

npm i https://pkg.pr.new/@trigger.dev/python@5ee6389

@trigger.dev/react-hooks

npm i https://pkg.pr.new/@trigger.dev/react-hooks@5ee6389

@trigger.dev/redis-worker

npm i https://pkg.pr.new/@trigger.dev/redis-worker@5ee6389

@trigger.dev/rsc

npm i https://pkg.pr.new/@trigger.dev/rsc@5ee6389

@trigger.dev/schema-to-json

npm i https://pkg.pr.new/@trigger.dev/schema-to-json@5ee6389

@trigger.dev/sdk

npm i https://pkg.pr.new/@trigger.dev/sdk@5ee6389

commit: 5ee6389