Paginate GET /api/v1/messages with limit and an opaque cursor

[awizemann]

Part 1 of 2 from #11 (pagination only, per your contract). Spec-first companion: HQBase/hqbase-site#10

Summary

• GET /api/v1/messages accepts limit (integer 1–100, default 100) and an opaque, versioned cursor over (activity_at, id).
• Orders by COALESCE(received_at, sent_at, created_at) DESC, id DESC; fetches limit + 1 to detect a next page.
• Body is unchanged (JSON array of MessageSummary). When another page exists the response carries Link: <absolute-url>; rel="next" preserving mailboxId, folder, search, limit plus cursor; no Link on the final page.
• Malformed limit/cursor → 400 INVALID_LIMIT / 400 INVALID_CURSOR in the standard error envelope. The mailbox-access predicate is built independently of the cursor and applies to every page.
• Message cursors carry their own version tag ("m1"), so a conversation cursor is rejected here and vice versa. The base64url keyset codec is factored into worker/features/messages/keyset-cursor.ts and conversations were moved onto it — wire format and INVALID_CONVERSATION_CURSOR behaviour are byte-for-byte unchanged (a shipped path, so worth a glance in review).
• Not included: updatedSince, a changes endpoint, or a change-journal migration.
• The web app and MCP tool are untouched (listMessages still returns the array and delegates to the new page query).

Index

EXPLAIN QUERY PLAN on a 200k-row replica showed USE TEMP B-TREE FOR ORDER BY for every filter shape (existing indexes are (col, created_at)), so migration 0012_message_activity_index.sql adds expression indexes on the activity order — one each for unfiltered, mailbox-filtered and folder-filtered listings — after which list, keyset and single-mailbox plans are index-served with no temp B-tree, and that holds on a fresh database without ANALYZE.

Two things I'd like your call on: (a) three indexes is 3 extra B-tree writes per inserted message; messages_folder_activity_idx + messages_mailbox_activity_idx cover every shape the web app issues, and messages_activity_idx only earns its keep for a multi-mailbox listing with no folder — happy to trim to two. (b) A multi-value mailbox_id IN (…) with no folder filter still sorts via a temp B-tree unless sqlite_stat1 exists (SQLite won't sort-merge across IN values without stats); D1 never runs ANALYZE on its own. I did not add ANALYZE to the migration.

Tests

Integration (test/integration/worker/mail-api.test.ts, new "message pagination" block): equal activity timestamps split across a page boundary (exact per-page contents asserted, so a missing id tiebreak or an off-by-one shows as a duplicate/dropped row); no Link on the final page; filter preservation in the next URL, then followed; an unreadable mailbox never appears on any page; a hand-built cursor pointing into an unreadable mailbox resumes correctly without leaking it; default and cap of 100; invalid limits (0, 101, abc, -1, 1.5, empty) → INVALID_LIMIT; malformed cursor and a genuine conversation cursor → INVALID_CURSOR.
Migration coverage: message-activity-index.test.ts (upgrade: plan uses a temp B-tree at 0011, does not after 0012; re-apply idempotent) and local-reset.test.ts (fresh install: the three indexes exist).

OpenAPI edited surgically (limit, cursor, Link header, 400 codes); Postman regenerated and verified. Agent Skill text updated to describe message cursors.

Validation

pnpm check: biome, typecheck, api:check, integration (11 files / 60 tests), coverage, architecture, build pass. Only failure is the pre-existing use-draft-autosave.test.tsx localStorage case under Node 26.5 (reproduces on pristine main). Architecture check now warns that queries.ts is over 300 lines (still passes); say if you'd like the paging query split into its own module.

🤖 Generated with Claude Code

[bermanto]

Thanks for this careful implementation. The pagination contract, access filtering, cursor versioning, and integration coverage look good. I found one focused blocker in the index migration: the default multi-mailbox query can still build a temporary order B-tree because the planner has no statistics for the new indexes. Please add PRAGMA optimize and cover the no-folder, multi-mailbox production query shape in the migration plan test. I would keep all three indexes; the global activity index is useful for broad unfiltered listings once the planner can choose it.

[awizemann]

Done in the latest commit:

• migrations/0012_message_activity_index.sql now ends with PRAGMA optimize; (with a comment on why).
• message-activity-index.test.ts seeds both readable mailboxes (400 rows) and adds the production default shape — mailbox_id IN (...) with no folder filter — to the after-migration assertions. I verified it discriminates: removing the pragma makes that assertion fail (USE TEMP B-TREE FOR ORDER BY returns); with it, all four shapes are index-served.
• All three indexes kept, as you suggested.

pnpm check locally: everything green except the pre-existing use-draft-autosave.test.tsx localStorage case under Node 26.5. The earlier quality-windows failure was two timeouts in users.test.ts (unrelated to this change) — flagging in case that job is still flaky for you.