Add cursor pagination to /flowsheet/search (step 3)#473
Merged
Conversation
Cursor mode keeps every page O(limit) regardless of depth — OFFSET grows linearly because Postgres still scans and discards skipped rows. Pairs cleanly with the flowsheet_track_add_time_idx partial index from migration 0050. Cursor only applies to sort=date. Other sorts (artist, song, dj) keep using offset because their sort columns are not unique and there is no compound index supporting a (sort_col, id) cursor — and date is the default and the only sort the recent-tracks landing page uses. Cursor is compound (add_time, id) rather than add_time alone because the legacy ETL backfilled many rows with batch-import timestamps that share the same add_time value to the microsecond — a single-column cursor would silently drop those rows on page boundaries. Encoding format is ISO_timestamp_id; malformed cursors return 400. Both response shapes coexist for backward compat: totalPages is always returned, nextCursor is added only when cursor mode produced a full page. The dj-site useLazySearchPlaylistsQuery currently reads totalPages and can migrate to nextCursor when convenient — that is a separate follow-up PR. Adds parseCursor / encodeCursor unit tests including round-trip and a comprehensive set of malformed-cursor cases. Adds searchFlowsheet tests covering: nextCursor presence on full page, omission on short page, omission in offset mode, omission when sort is not date, total still returned in cursor mode. Adds controller tests for valid forwarding, undefined-when-missing, and 400 on malformed cursor. Updates the design doc step 3 section to reflect the compound-cursor decision and the date-only sort coverage. Implements step 3 from docs/playlist-search/README.md.
This was referenced Apr 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cursorquery parameter toGET /flowsheet/search. When provided withsort=date, the WHERE switches fromOFFSETto a row-value comparison(add_time, id) < (cursor_time, cursor_id)(or>for asc) so each page costs O(limit) regardless of depth.(add_time, id)rather thanadd_timealone — the legacy ETL backfilled many rows with batch-import timestamps that share the sameadd_timeto the microsecond, and a single-column cursor would silently drop those rows on page boundaries.artist,song,dj) ignore cursor and continue using offset because their sort columns are not unique and no compound index supports a(sort_col, id)cursor.nextCursorreturned only when cursor mode is active and the page came back full.totalPagesstays in the response always — both shapes coexist for the dj-site backward-compat window.${ISO_timestamp}_${id}. Malformed cursor →400.Implements step 3 from
docs/playlist-search/README.md.Test plan
parseCursor/encodeCursorcodec tests including round-trip and a comprehensive set of malformed-cursor casesnextCursoron full page, omission on short page, omission in offset mode, omission when sort is not date, total still returned in cursor modenextCursornpm run typecheckcleannpm run lintclean (warnings unchanged from main)npx prettier --checkclean on touched filesCloses #472