Skip to content

feat: API improvements#6

Merged
isTravis merged 2 commits into
mainfrom
tr/api-improvements
May 1, 2026
Merged

feat: API improvements#6
isTravis merged 2 commits into
mainfrom
tr/api-improvements

Conversation

@isTravis
Copy link
Copy Markdown
Member

@isTravis isTravis commented May 1, 2026

Infrastructure improvements to handle large collections and reduce server load.

Changes

  • Chunked upload protocol — New multi-request upload flow for large version pushes (>100MB). Clients start a session, append records in batches (up to 10K per request), then finalize. Streaming hash computation and Postgres temp tables keep memory bounded during finalize.
  • Cursor-based pagination — Records endpoint supports ?after=recordId keyset pagination alongside existing offset pagination. Eliminates slow OFFSET scans on large collections.
  • CDN file redirects — File download endpoint now returns a 301 to assets.underlay.org instead of proxying bytes through the app server. Visibility checks still run before redirect.
  • Performance indexes — Added indexes on collections(account_id), records(version_id, type), and version_files(file_hash).
  • Migration lockingmigrate.ts uses a Postgres advisory lock to prevent race conditions when multiple replicas start simultaneously.

New

  • Tables: upload_sessions, upload_records
  • Route: src/api/routes/uploads.ts
  • Migrations: 0002_harsh_madrox.sql, 0003_cool_leper_queen.sql

@isTravis isTravis merged commit e1d177b into main May 1, 2026
@isTravis isTravis deleted the tr/api-improvements branch May 1, 2026 04:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant