feat(rust-client): Add Multipart Upload API by lcian · Pull Request #468 · getsentry/objectstore

lcian · 2026-05-08T15:59:11Z

⚠️ Stacked on #463

Implements a multipart upload API for the Rust client.
Instead of going for a low-level API where calls match exactly HTTP calls as I've planned initially, I've decided to abstract some things, for example, initiating a multipart upload returns a MultipartUpload that represents it, much like the existing Session we have.
Please look at the README to see the intended usage.

Also makes a couple changes that aren't strictly related to the multipart upload impl:

Moves multipart upload-related types from objectstore-server to objectstore-types, to make them usable from the client too, given that they both deal with the wire format.
Makes the server return a vec of completed parts, each with their own part_number when calling ListParts, as opposed to a BTreeMap.
Makes Usecase::compression take an Option<Compression>. This is a breaking change that makes it possible to disable default-compression on a usecase, which wasn't possible before.

Close FS-357

codecov · 2026-05-08T16:09:56Z

Codecov Report

❌ Patch coverage is 77.14286% with 72 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.56%. Comparing base (b84e11f) to head (bb111e4).

Files with missing lines	Patch %	Lines
clients/rust/src/multipart.rs	71.09%	61 Missing ⚠️
objectstore-types/src/multipart.rs	74.19%	8 Missing ⚠️
objectstore-server/src/endpoints/common.rs	0.00%	1 Missing ⚠️
objectstore-service/src/backend/tiered.rs	66.66%	1 Missing ⚠️
objectstore-service/src/error.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #468      +/-   ##
==========================================
- Coverage   87.76%   87.56%   -0.21%     
==========================================
  Files          80       81       +1     
  Lines       12621    12896     +275     
==========================================
+ Hits        11077    11292     +215     
- Misses       1544     1604      +60

Components	Coverage Δ
Rust Backend	`92.33% <73.80%> (-0.06%)`	⬇️
Rust Client	`79.70% <77.65%> (-0.08%)`	⬇️
Python Client	`86.36% <ø> (ø)`

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

matt-codecov

ran out of steam before i could finish reviewing, sorry

matt-codecov

accept w comments (including my comments last week)

The test was inheriting the usecase default (zstd) compression but uploading raw uncompressed parts, causing "Unknown frame descriptor" on read-back.

Instead of disabling compression, send properly zstd-compressed parts so the test exercises the multiple_members concatenated-frame decompression path that the client uses for multipart objects.

Allow callers to disable compression at the Usecase level by passing None to with_compression(). Previously the method only accepted a Compression value, making it impossible to set "no compression" as the usecase-wide default. The internal field and compression() accessor are now Option<Compression>. The default remains Compression::Zstd. Per-operation builders are unaffected since they already accepted Option<Compression>. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

… test The full_upload_uncompressed test was sending zstd-compressed parts while the upload metadata had compression=None, so GET returned the raw compressed bytes instead of the expected plaintext.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit b1e6a42. Configure here.}

These were carried over from the old base branch during the restack and downgraded several Python dependencies.

Moves the validated `UploadId` newtype and `PartNumber` (`NonZeroU32`) type alias from `objectstore-service` to `objectstore-types` so they can be shared between the server and Rust client. Updates all wire format types to use the stricter types, removing the need for conversions at the server endpoint boundary.

The client methods (put, put_stream, put_read, resume_multipart_upload) accept u32 for part numbers and String for upload IDs, converting to the stricter internal types (PartNumber, UploadId) at the boundary. This keeps the public API simple while maintaining type safety internally.

Replace InvalidArgument with InvalidPartNumber and InvalidUploadId.

Previously fell through to the catch-all 500 Internal Server Error.

⚠️ Stacked on #468 Adds an optional `multipart` config block for stresstest workloads that exercises the new multipart uploads feature. When enabled, writes for objects >1 KiB use multipart uploads with the configured part size and concurrency. This can be used in both `weighted` and `throughput` mode, but not in `batch` mode, as that wouldn't make any sense. Also adds a `README.md` that documents the meaning of all stresstest options and when they can/should be used. Part of FS-343 Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

⚠️ Stacked on #468 Adds a Multipart Upload API to the Python client. The API is basically the same as the one implemented in #468 for the Rust client. Close FS-340

lcian changed the base branch from main to lcian/feat/multipart-upload-endpoints May 8, 2026 16:09

This comment was marked as outdated.

Sign in to view

This comment was marked as spam.

Sign in to view

lcian changed the title ~~lcian/feat/multipart upload rust client~~ feat(rust-client): Add Multipart Upload API May 11, 2026

cursor Bot reviewed May 11, 2026

View reviewed changes

Comment thread clients/rust/src/multipart.rs

lcian commented May 11, 2026

View reviewed changes

Comment thread clients/rust/src/multipart.rs

lcian marked this pull request as ready for review May 11, 2026 13:10

lcian requested a review from a team as a code owner May 11, 2026 13:10

This comment was marked as spam.

Sign in to view

lcian mentioned this pull request May 11, 2026

feat(stresstest): Add multipart capability #470

Merged

lcian marked this pull request as draft May 12, 2026 14:25

lcian marked this pull request as ready for review May 12, 2026 14:27

This comment was marked as outdated.

Sign in to view

lcian mentioned this pull request May 15, 2026

feat(py-client): Add Multipart Upload API #476

Merged

lcian requested a review from matt-codecov May 15, 2026 16:37

lcian force-pushed the lcian/feat/multipart-upload-endpoints branch from 5babbfb to c33ba6f Compare May 15, 2026 16:57

lcian force-pushed the lcian/feat/multipart-upload-rust-client branch from 93557e1 to 84181ed Compare May 15, 2026 17:01

matt-codecov reviewed May 22, 2026

View reviewed changes

Comment thread clients/rust/src/get.rs

Comment thread clients/rust/src/multipart.rs

Comment thread clients/rust/src/multipart.rs

Comment thread clients/rust/src/multipart.rs

matt-codecov self-requested a review May 22, 2026 01:27

matt-codecov approved these changes May 26, 2026

View reviewed changes

Comment thread clients/rust/src/multipart.rs

lcian force-pushed the lcian/feat/multipart-upload-endpoints branch from ab5d578 to a2496b3 Compare May 29, 2026 08:20

Base automatically changed from lcian/feat/multipart-upload-endpoints to main May 29, 2026 09:28

lcian added 4 commits May 29, 2026 11:50

wip

c38d880

wip

393ea96

wip

8d0a49d

wip

c20008f

lcian and others added 19 commits May 29, 2026 11:50

fix(test): set compression(None) in full_upload_flow multipart test

8f51b3f

The test was inheriting the usecase default (zstd) compression but uploading raw uncompressed parts, causing "Unknown frame descriptor" on read-back.

fix(test): pre-compress parts in full_upload_flow multipart test

0571711

Instead of disabling compression, send properly zstd-compressed parts so the test exercises the multiple_members concatenated-frame decompression path that the client uses for multipart objects.

revert unrelated changes

5b7e1a0

revert unrelated changes

b5ee8fc

improve

7ed2130

improve

adb0317

improve

0452d6e

improve

59b2dec

fix: correct broken intra-doc link in multipart.rs

697d21e

improve

7e5fdf6

fix(rust-client): don't zstd-compress parts in uncompressed multipart…

d0518be

… test The full_upload_uncompressed test was sending zstd-compressed parts while the upload metadata had compression=None, so GET returned the raw compressed bytes instead of the expected plaintext.

improve

cd8ba49

improve

9b399c1

improve

d86cecb

improve

b1a695f

improve

d84038f

fix

a2515a6

add comment on decoder.multiple_members(true)

b1e6a42

lcian force-pushed the lcian/feat/multipart-upload-rust-client branch from 3565150 to b1e6a42 Compare May 29, 2026 09:51

cursor Bot reviewed May 29, 2026

View reviewed changes

Comment thread objectstore-server/src/endpoints/multipart.rs

lcian added 5 commits May 29, 2026 11:57

revert spurious pyproject.toml and uv.lock changes

175e3fc

These were carried over from the old base branch during the restack and downgraded several Python dependencies.

ref(rust-client): use specific error variants for validation failures

59da6dd

Replace InvalidArgument with InvalidPartNumber and InvalidUploadId.

fix(server): map InvalidUploadId to 400 Bad Request

bb111e4

Previously fell through to the catch-all 500 Internal Server Error.

lcian merged commit 8e5faf7 into main May 29, 2026
25 checks passed

lcian deleted the lcian/feat/multipart-upload-rust-client branch May 29, 2026 11:04

lcian added a commit that referenced this pull request May 29, 2026

feat(py-client): Add Multipart Upload API (#476)

324009d

⚠️ Stacked on #468 Adds a Multipart Upload API to the Python client. The API is basically the same as the one implemented in #468 for the Rust client. Close FS-340

Uh oh!

Conversation

lcian commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

Uh oh!

This comment was marked as spam.

This comment was marked as spam.

Uh oh!

Uh oh!

This comment was marked as spam.

This comment was marked as outdated.

Uh oh!

matt-codecov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matt-codecov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lcian commented May 8, 2026 •

edited

Loading

codecov Bot commented May 8, 2026 •

edited

Loading