Skip to content

Reduce S3 Calls (GSI-2490)#241

Merged
TheByronHimes merged 13 commits into
mainfrom
feature/reduce_s3_calls_GSI-2490
Jul 16, 2026
Merged

Reduce S3 Calls (GSI-2490)#241
TheByronHimes merged 13 commits into
mainfrom
feature/reduce_s3_calls_GSI-2490

Conversation

@TheByronHimes

@TheByronHimes TheByronHimes commented Jul 8, 2026

Copy link
Copy Markdown
Member

Re-evaluated every S3 provider operation and removed defensive existence checks where the main S3 call already fails with a translatable error.

Removed prechecks

  • delete_bucket and list_all_object_ids rely on NoSuchBucket
  • list_parts, abort_multipart_upload, and complete_multipart_upload rely on NoSuchUpload
  • get_object_etag/get_object_size/get_object_metadata rely on head_object failing

Optimized

  • _assert_object_(not_)exists now checks the object first and only checks the bus one call instead of two

Kept (with comments explaining why)

  • create_bucket - S3 silently succeeds re-creating an owned bucket, so we let the app handle that error
  • Presigned URL ops - signing is local, nothing server-side would raise if the upload/bucket didn't exist
  • init_multipart_upload - to prevent duplicate uploads
  • delete_object - so the application decides how to interpret missing objects
  • copy_object dest check - to prevent overwriting things by accident

Note for near future

We should consider removing MultipleActiveUploadsError. The built-in check for multiple active uploads doesn't give us much. The methods already target specific uploads via upload_id, and if an application needs enforce upload exclusivity then that should be done application-side. That would mandate a major version bump though.

Version bumped to 8.5.0

@TheByronHimes
TheByronHimes marked this pull request as ready for review July 8, 2026 21:00
@TheByronHimes
TheByronHimes requested a review from mephenor July 8, 2026 21:02
@coveralls

coveralls commented Jul 8, 2026

Copy link
Copy Markdown

Coverage Report for CI Build 29416503475

Warning

No base build found for commit c26d86f on main.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 93.664%

Details

  • Patch coverage: 1 uncovered change across 1 file (28 of 29 lines covered, 96.55%).

Uncovered Changes

File Changed Covered %
src/hexkit/providers/s3/provider/objstorage.py 16 15 93.75%
Total (3 files) 29 28 96.55%

Coverage Regressions

Requires a base build to compare against. How to fix this →


Coverage Stats

Coverage Status
Relevant Lines: 3977
Covered Lines: 3725
Line Coverage: 93.66%
Coverage Strength: 0.94 hits per line

💛 - Coveralls

@TheByronHimes
TheByronHimes force-pushed the feature/reduce_s3_calls_GSI-2490 branch from e5dd2b6 to 21a3e2d Compare July 13, 2026 13:03

@mephenor mephenor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I just some additional places that could use slight improvement.

Do the existing tests not cover all the failure modes for the provider?
If so, should we extend them?

Comment thread src/hexkit/providers/s3/provider/objstorage.py
Comment thread src/hexkit/providers/s3/provider/objstorage.py Outdated
Comment thread src/hexkit/providers/s3/provider/objstorage.py Outdated
Comment thread src/hexkit/providers/s3/provider/objstorage.py
@TheByronHimes
TheByronHimes requested a review from mephenor July 15, 2026 12:53
@TheByronHimes
TheByronHimes merged commit ce688ed into main Jul 16, 2026
8 checks passed
@TheByronHimes
TheByronHimes deleted the feature/reduce_s3_calls_GSI-2490 branch July 16, 2026 07:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants