Skip to content

fix/issue 303 - #326

Merged
duyet merged 1 commit into
mainfrom
fix/issue-303
Aug 2, 2026
Merged

fix/issue 303#326
duyet merged 1 commit into
mainfrom
fix/issue-303

Conversation

@duyet

@duyet duyet commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Summary by Sourcery

Implement real cursor-based pagination for the files API, returning a meaningful next_page token and aligning pagination behavior across services and adapters.

Bug Fixes:

  • Ensure AMA files list returns a real continuation token via next_page that correctly walks all pages and terminates instead of hardcoding null.

Enhancements:

  • Introduce shared cursor-based pagination over (created_at, id) for file listing, supporting both opaque page tokens and id-based anchors in the service and repository ports.
  • Align Cloudflare and Node files list routes to use the new listPage API, including consistent first_id/last_id anchors and has_more semantics.
  • Update SQL and in-memory file repositories to honor the new PageCursor-based seek semantics with stable ordering and adapter parity.

Tests:

  • Add integration tests validating next_page-driven pagination and SDK auto-pagination traverse multiple pages and terminate correctly.

`next_page` was hardcoded null on both the Cloudflare and self-host Node
list routes, so a client following the token stopped after page 1.

- Wire `next_page` to the shared opaque (created_at, id) DESC cursor from
  packages/shared/src/pagination.ts via a new FileService.listPage.
- Accept the token back as `?page_token=` (or `?page=`).
- Replace the files repo's lexicographic id comparison with a real
  (created_at, id) seek, so `after_id` / `before_id` also walk the true
  ordering. The service resolves an id anchor to the same cursor, and an
  anchor naming no row now returns an empty page instead of silently
  restarting at page 1 (which looped SDK auto-pagination).
- Order by (created_at, id) so pages never overlap within one millisecond.
- Fold session-outputs R2 rows into the first page only; keep first_id /
  last_id pointing at D1 rows so the SDK's id pager stays valid.
- Node's read-only route now paginates identically.

Closes #303
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @duyet, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@duyet, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 41 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: aefcb20b-04b3-4665-9fbc-880f8881e387

📥 Commits

Reviewing files that changed from the base of the PR and between f18388c and 894f3d4.

📒 Files selected for processing (7)
  • apps/main-node/src/index.ts
  • apps/main/src/routes/files.ts
  • packages/files-store/src/adapters/sql-file-repo.ts
  • packages/files-store/src/ports.ts
  • packages/files-store/src/service.ts
  • packages/files-store/src/test-fakes.ts
  • test/integration/ama-sdk-files.test.ts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai

sourcery-ai Bot commented Aug 2, 2026

Copy link
Copy Markdown

Reviewer's Guide

Refactors AMA files listing to use a shared (created_at, id) cursor-based pagination scheme, wiring real continuation tokens (next_page) through the service, SQL/in‑memory repos, and both Cloudflare and Node HTTP routes, and adds integration tests to lock in SDK-compatible paging behavior.

Sequence diagram for cursor-based files listing with next_page tokens

sequenceDiagram
  actor Client
  participant CloudflareFilesRoute as CloudflareFilesRoute
  participant FileService as FileService
  participant SqlFileRepo as SqlFileRepo
  participant SharedPagination as SharedPagination

  Client->>CloudflareFilesRoute: GET /files (page_token)
  CloudflareFilesRoute->>FileService: listPage(tenantId, sessionId, cursor, beforeId, afterId, order, limit)
  FileService->>SharedPagination: decodeCursor(cursor)
  Note right of FileService: If cursor is invalid, decodeCursor returns undefined
  FileService->>SqlFileRepo: list(tenantId, { after, before, order, limit: fetchN(limit) })
  SqlFileRepo-->>FileService: rows
  FileService->>SharedPagination: trimPage(rows, limit)
  FileService->>SharedPagination: toCursorPage(items, cursorFromRow)
  SharedPagination-->>FileService: { items, nextCursor }
  FileService-->>CloudflareFilesRoute: page(items, nextCursor)
  CloudflareFilesRoute->>CloudflareFilesRoute: map items to ApiFileRecord
  CloudflareFilesRoute-->>Client: JSON { data, has_more, first_id, last_id, next_page }
Loading

File-Level Changes

Change Details Files
Introduce cursor-based paging in the files service and expose a new listPage API used by HTTP routes.
  • Add FileListArgs interface and refactor list() to delegate to listPage() for backwards-compatible item-only listing.
  • Implement listPage() to accept an optional cursor token, resolve before/after id anchors to shared PageCursor via anchor(), and return items plus nextCursor using shared helpers (decodeCursor, fetchN, trimPage, toCursorPage).
  • Enforce pagination contract where invalid id anchors short-circuit to an empty page, while stale opaque cursors restart from page 1.
  • Add private anchor() helper that reads an anchor file row and converts it into a PageCursor.
packages/files-store/src/service.ts
Align SQL and in-memory file repositories with the new PageCursor-based seek semantics over (created_at, id).
  • Change FileListOptions cursor fields from beforeId/afterId to before/after PageCursor and propagate type updates.
  • Update SqlFileRepo.list to build predicates using seek() over created_at and id, with direction depending on requested order, and order results by (created_at, id) in that direction.
  • Add a seek(PageCursor, forward) helper function implementing the (created_at, id) comparison via drizzle-orm SQL primitives.
  • Update InMemoryFileRepo.list to use seek() semantics for before/after filtering and to sort rows by created_at then id for deterministic ordering matching SQL.
  • Add a seek helper in the in-memory repo mirroring the SQL adapter’s seek logic.
packages/files-store/src/ports.ts
packages/files-store/src/adapters/sql-file-repo.ts
packages/files-store/src/test-fakes.ts
Wire real continuation tokens and id-based anchors through the Cloudflare worker files route, including interaction with synthesized session output rows.
  • Change the route to use FileService.listPage instead of list, passing through cursor (page_token/page), before_id, after_id, and order from query params while clamping limit server-side.
  • Compute has_more from presence of nextCursor, and surface next_page as that cursor (or null on last page) instead of a hardcoded null.
  • Limit synthesized session output rows (from R2) to the first page only when there’s no cursor/anchors, avoiding repetition across pages and keeping anchors tied to real D1 rows.
  • Set first_id/last_id based on the underlying D1 rows (anchors) when present, falling back to the returned data, to ensure SDK id-based pagination anchors are valid and ignore synthesized ids.
apps/main/src/routes/files.ts
Make the Node v1 /files route use the shared cursor paging contract with proper next_page tokens and id anchors.
  • Refactor Node route to call filesService.listPage, reading before_id, after_id, order, and cursor (page_token/page) from query params with limit enforcement.
  • Return data from page.items, has_more from presence of nextCursor, and next_page as the cursor or null on the final page.
  • Add first_id and last_id fields based on the returned items to support AMA SDK id-pager expectations, mirroring the Cloudflare route’s contract minus R2 synthesis.
apps/main-node/src/index.ts
Add integration tests to lock in SDK-compatible pagination behavior using next_page tokens and SDK auto-pagination over id-based anchors.
  • Add a test that seeds multiple files, walks pages via next_page using the HTTP wire format, and asserts that every page is reachable, terminates, and no file id is repeated, with has_more consistent with next_page.
  • Add a test that uses client.beta.files.list with a small limit to verify the SDK’s auto-pagination traverses more than one page, terminates, and does not duplicate ids, ensuring the route honors id-based paging (last_idafter_id).
test/integration/ama-sdk-files.test.ts

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@duyet
duyet merged commit d7d8045 into main Aug 2, 2026
6 checks passed
@duyet
duyet deleted the fix/issue-303 branch August 2, 2026 06:36
@duyet duyet mentioned this pull request Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants