Skip to content

perf(sync-server): fetch R2 payloads concurrently in pullItems#557

Merged
h4yfans merged 1 commit into
mainfrom
perf/pullitems-concurrent-r2
Jun 12, 2026
Merged

perf(sync-server): fetch R2 payloads concurrently in pullItems#557
h4yfans merged 1 commit into
mainfrom
perf/pullitems-concurrent-r2

Conversation

@h4yfans

@h4yfans h4yfans commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Replace the serial per-item R2 for await loop in pullItems with bounded-concurrency windows (R2_CONCURRENCY = 25): map over the already server_cursor-sorted rows in fixed-size windows, await Promise.all each window, concat in order. Cuts first-sync / large-pull latency from O(N) × per-object round-trips to overlapping reads, capped so a Worker never fires hundreds of simultaneous subrequests.
  • Behavior preserved: output stays ordered by server_cursor, nulls are filtered, and pullItems' signature/return type, toPullItemResponse, and the D1 batching loop are untouched.
  • Implements Plan 007.

Test plan

  • pnpm typecheck:sync-server → exit 0
  • pnpm test:sync-server → 634 pass (45 files), no flake
  • 3 new pullItems tests:
    • order preserved across concurrency windows even when R2 reads resolve out of order
    • unsupported-type rows filtered while surviving rows keep order
    • concurrency bounded — 1 < peak ≤ R2_CONCURRENCY (this assertion is red on the old serial loop, so it pins the new behavior)

Notes

  • Docs gate: pushed with MEMRY_DOCS_IMPACT_SKIP=1. This is an internal performance refactor with identical inputs/outputs/ordering — no API, contract, behavior, or documented-surface change, so there is nothing to update under apps/docs/src.

Closes #548

@vercel

vercel Bot commented Jun 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
memry-docs Building Building Preview, Comment Jun 12, 2026 10:02pm
memrynote-landing Building Building Preview, Comment Jun 12, 2026 10:02pm

@h4yfans h4yfans marked this pull request as ready for review June 12, 2026 22:02
@h4yfans h4yfans merged commit bd6067d into main Jun 12, 2026
12 of 18 checks passed
@github-actions

Copy link
Copy Markdown

React Doctor

React Doctor found 3 files changed in this pull request, but none matched the files covered by its enabled checks.

Scope: 3 files changed on perf/pullitems-concurrent-r2 vs. main.

View workflow run

Generated by React Doctor. Questions? Contact founders@million.dev.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fetch R2 payloads concurrently (bounded) in pullItems instead of one round-trip per item

1 participant