fix(ategcs): stop calling CreateBucket on every PutObject#22
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
|
Hmm, perhaps we should make the kind environment also pre-create the bucket on install and drop this entirely. In general we want the CI environment to look like production where possible. |
Yeah that would make sense; i wasn't sure what the general idea was but that works as well if we want to go that route. |
s3Client.PutObject called CreateBucket before every upload. Convenient against local dev backends like rustfs/minio where buckets may not pre-exist, but against managed S3-compatible backends the caller typically lacks s3:CreateBucket, so each snapshot upload paid for a 403 in added latency and audit-log noise. For local kind dev, add a one-shot Job alongside the rustfs Deployment that creates the ate-snapshots bucket once at install time.
0547bfb to
11e8bd0
Compare
|
Benjamin Elder (@BenTheElder) I dropped the create call and added it to kind setup if we want to go that way instead. |
Benjamin Elder (BenTheElder)
left a comment
There was a problem hiding this comment.
Thank you!
Unfortunately as far as I know we require the CLA currently.
Otherwise aside from one minor comment I am +1
|
Working on getting that resolved! |
Co-authored-by: Benjamin Elder <bentheelder@google.com>
|
CLA Signed |
d8865a4
into
agent-substrate:main
|
Thanks! |
Summary
s3Client.PutObjectcalledCreateBucketbefore every upload. Convenient against local dev backends like rustfs/minio where buckets may not pre-exist, but against managed S3-compatible backends the caller typically lackss3:CreateBucket, so each snapshot upload paid for a 403 in added latency and audit-log noise.For local kind dev, add a one-shot Job alongside the rustfs Deployment that creates the
ate-snapshotsbucket once at install time.Validation
Verified locally with a kind cluster:
manifests/ate-install/kind/rustfs.yamlrustfs-bucket-initJob retried once while rustfs came up, then createdate-snapshotsaws s3api list-bucketsagainst rustfs returnsate-snapshots