Skip to content

fix(ategcs): stop calling CreateBucket on every PutObject#22

Merged
Benjamin Elder (BenTheElder) merged 2 commits into
agent-substrate:mainfrom
coreweave:bjacobs/s3-skip-bucket-create
May 21, 2026
Merged

fix(ategcs): stop calling CreateBucket on every PutObject#22
Benjamin Elder (BenTheElder) merged 2 commits into
agent-substrate:mainfrom
coreweave:bjacobs/s3-skip-bucket-create

Conversation

@brandonrjacobs
Copy link
Copy Markdown
Contributor

@brandonrjacobs Brandon Jacobs (brandonrjacobs) commented May 20, 2026

Summary

s3Client.PutObject called CreateBucket before every upload. Convenient against local dev backends like rustfs/minio where buckets may not pre-exist, but against managed S3-compatible backends the caller typically lacks s3:CreateBucket, so each snapshot upload paid for a 403 in added latency and audit-log noise.

For local kind dev, add a one-shot Job alongside the rustfs Deployment that creates the ate-snapshots bucket once at install time.

Validation

Verified locally with a kind cluster:

  • Applied namespace + manifests/ate-install/kind/rustfs.yaml
  • rustfs-bucket-init Job retried once while rustfs came up, then created ate-snapshots
  • aws s3api list-buckets against rustfs returns ate-snapshots

@google-cla
Copy link
Copy Markdown

google-cla Bot commented May 20, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@BenTheElder
Copy link
Copy Markdown
Collaborator

Hmm, perhaps we should make the kind environment also pre-create the bucket on install and drop this entirely.

In general we want the CI environment to look like production where possible.

@brandonrjacobs
Copy link
Copy Markdown
Contributor Author

Hmm, perhaps we should make the kind environment also pre-create the bucket on install and drop this entirely.

In general we want the CI environment to look like production where possible.

Yeah that would make sense; i wasn't sure what the general idea was but that works as well if we want to go that route.

s3Client.PutObject called CreateBucket before every upload. Convenient
against local dev backends like rustfs/minio where buckets may not
pre-exist, but against managed S3-compatible backends the caller
typically lacks s3:CreateBucket, so each snapshot upload paid for a 403
in added latency and audit-log noise.

For local kind dev, add a one-shot Job alongside the rustfs Deployment
that creates the ate-snapshots bucket once at install time.
@brandonrjacobs Brandon Jacobs (brandonrjacobs) changed the title feat(ategcs): gate per-Put CreateBucket behind opt-in flag fix(ategcs): drop per-Put CreateBucket from S3 backend May 20, 2026
@brandonrjacobs
Copy link
Copy Markdown
Contributor Author

Benjamin Elder (@BenTheElder) I dropped the create call and added it to kind setup if we want to go that way instead.

@brandonrjacobs Brandon Jacobs (brandonrjacobs) changed the title fix(ategcs): drop per-Put CreateBucket from S3 backend fix(ategcs): stop calling CreateBucket on every PutObject May 20, 2026
@brandonrjacobs Brandon Jacobs (brandonrjacobs) marked this pull request as ready for review May 20, 2026 21:22
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Unfortunately as far as I know we require the CLA currently.

Otherwise aside from one minor comment I am +1

Comment thread manifests/ate-install/kind/rustfs.yaml Outdated
@brandonrjacobs
Copy link
Copy Markdown
Contributor Author

Working on getting that resolved!

Co-authored-by: Benjamin Elder <bentheelder@google.com>
@brandonrjacobs
Copy link
Copy Markdown
Contributor Author

CLA Signed

@BenTheElder Benjamin Elder (BenTheElder) merged commit d8865a4 into agent-substrate:main May 21, 2026
4 checks passed
@BenTheElder
Copy link
Copy Markdown
Collaborator

Thanks!

@BenTheElder Benjamin Elder (BenTheElder) added the bug Something isn't working / bugfixes label May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working / bugfixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants