Skip to content

fix(storage-*): redundant re-uploads when file was already uploaded by the client and add comprehensive E2E tests for s3 and vercel-blob#16115

Merged
r1tsuu merged 5 commits intomainfrom
test/e2e-clientUploads
Apr 1, 2026
Merged

fix(storage-*): redundant re-uploads when file was already uploaded by the client and add comprehensive E2E tests for s3 and vercel-blob#16115
r1tsuu merged 5 commits intomainfrom
test/e2e-clientUploads

Conversation

@r1tsuu
Copy link
Copy Markdown
Member

@r1tsuu r1tsuu commented Mar 31, 2026

Summary

  • Restructures clientUploads.config.tsclient-uploads/config.ts subdirectory for both storage-s3 and storage-vercel-blob, enabling pnpm run dev storage-s3/client-uploads to work correctly
  • Moves integration tests into the same subdirectory
  • Adds a MediaContainer collection (hasMany upload → media) to support bulk upload drawer testing
  • Adds E2E test suites (e2e.spec.ts) for both adapters that verify via network interception that file bytes never reach the Payload server - only a JSON descriptor is POSTed after the browser uploads directly to S3/Vercel Blob
  • Configures S3 CORS on the localstack bucket via an init script (test/localstack-init/ready.d/01-s3-cors.sh) mounted in docker-compose.yml, enabling cross-origin browser PUTs from localhost:3000 to localhost:4566

Change in packages/plugin-cloud-storage/src/hooks/afterChange.ts:

When a file was client-uploaded (browser → storage directly), the afterChange hook called adapter.handleUpload on the original file a second time, triggering a redundant server-side upload. For Vercel Blob this caused a BlobUnknownError (file already exists); for S3 it caused a silent duplicate PUT.

Fix: filter out files where file.clientUploadContext is set before calling handleUpload. Image resize variants generated server-side by Payload/sharp do not have clientUploadContext and continue to be uploaded server-side as before - size variants still work with this

E2E Testing

Each suite (storage-s3/client-uploads/e2e.spec.ts, storage-vercel-blob/client-uploads/e2e.spec.ts) covers three scenarios:

  1. Single upload completes successfully via the admin UI
  2. Network verification (single) — intercepts all browser requests; asserts no request to localhost:3000 carries file bytes (content-length > 10 KB), and at least one PUT/request goes to the storage endpoint - important as on Vercel if we have a request like the upload will fail.
  3. Network verification (field bulk upload) — same assertions for the bulk-upload drawer opened from a hasMany upload relationship field (MediaContainer.files); asserts two items appear in the field after the drawer closes
  4. Network verification (list view bulk upload) — same network assertions for the "Bulk Upload" button in the collection list-view header (#bulk-upload-drawer-slug-1), which is a separate entry point from the field-level drawer

… the client and add comprehensive E2E test suites for `s3` and `vercel-blob`
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 31, 2026

📦 esbuild Bundle Analysis for payload

This analysis was generated by esbuild-bundle-analyzer. 🤖
This PR introduced no changes to the esbuild bundle! 🙌

@r1tsuu r1tsuu merged commit a6735f3 into main Apr 1, 2026
312 of 314 checks passed
@r1tsuu r1tsuu deleted the test/e2e-clientUploads branch April 1, 2026 16:15
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 8, 2026

🚀 This is included in version v3.82.0

milamer pushed a commit to milamer/payload that referenced this pull request Apr 20, 2026
…y the client and add comprehensive E2E tests for `s3` and `vercel-blob` (payloadcms#16115)

## Summary

- Restructures `clientUploads.config.ts` → `client-uploads/config.ts`
subdirectory for both `storage-s3` and `storage-vercel-blob`, enabling
`pnpm run dev storage-s3/client-uploads` to work correctly
- Moves integration tests into the same subdirectory
- Adds a `MediaContainer` collection (hasMany upload → media) to support
bulk upload drawer testing
- Adds E2E test suites (`e2e.spec.ts`) for both adapters that verify via
network interception that file bytes never reach the Payload server -
only a JSON descriptor is POSTed after the browser uploads directly to
S3/Vercel Blob
- Configures S3 CORS on the localstack bucket via an init script
(`test/localstack-init/ready.d/01-s3-cors.sh`) mounted in
`docker-compose.yml`, enabling cross-origin browser PUTs from
`localhost:3000` to `localhost:4566`


Change in **`packages/plugin-cloud-storage/src/hooks/afterChange.ts`**:

When a file was client-uploaded (browser → storage directly), the
`afterChange` hook called `adapter.handleUpload` on the original file a
second time, triggering a redundant server-side upload. For Vercel Blob
this caused a `BlobUnknownError` (file already exists); for S3 it caused
a silent duplicate PUT.

Fix: filter out files where `file.clientUploadContext` is set before
calling `handleUpload`. Image resize variants generated server-side by
Payload/sharp do not have `clientUploadContext` and continue to be
uploaded server-side as before - **size variants still work with this**

## E2E Testing

Each suite (`storage-s3/client-uploads/e2e.spec.ts`,
`storage-vercel-blob/client-uploads/e2e.spec.ts`) covers three
scenarios:

1. Single upload completes successfully via the admin UI
2. **Network verification (single)** — intercepts all browser requests;
asserts no request to `localhost:3000` carries file bytes
(content-length > 10 KB), and at least one PUT/request goes to the
storage endpoint - important as on Vercel if we have a request like the
upload will fail.
3. **Network verification (field bulk upload)** — same assertions for
the bulk-upload drawer opened from a `hasMany` upload relationship field
(`MediaContainer.files`); asserts two items appear in the field after
the drawer closes
4. **Network verification (list view bulk upload)** — same network
assertions for the "Bulk Upload" button in the collection list-view
header (`#bulk-upload-drawer-slug-1`), which is a separate entry point
from the field-level drawer

---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
  - https://app.asana.com/0/0/1213884972829065
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

storage-vercel-blob: Upload fails sometimes when using clientUpload: true

2 participants