Skip to content

fix: s3 bucket operations in V2#5803

Merged
zhaoqizqwang merged 4 commits intoaws:master-v2from
zhaoqizqwang:v2-bucket-fix
Apr 29, 2026
Merged

fix: s3 bucket operations in V2#5803
zhaoqizqwang merged 4 commits intoaws:master-v2from
zhaoqizqwang:v2-bucket-fix

Conversation

@zhaoqizqwang
Copy link
Copy Markdown
Collaborator

Add ExpectedBucketOwner to S3 data-plane calls

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Add ExpectedBucketOwner to S3 data-plane calls
@zhaoqizqwang zhaoqizqwang requested a review from a team as a code owner April 28, 2026 22:07
@zhaoqizqwang zhaoqizqwang requested a review from chad119 April 28, 2026 22:07
@zhaoqizqwang
Copy link
Copy Markdown
Collaborator Author

The failed tests are flaky and not caused by code changes in this PR:

1. [gw45] [ 21%] FAILED tests/integ/test_huggingface.py::test_huggingface_training[4.56.2-2.8.0]

tests/integ/test_knn.py::test_async_knn_classifier

2. [gw93] [ 22%] FAILED tests/integ/test_huggingface.py::test_framework_processing_job_with_deps[4.56.2-2.8.0]

3. [gw26] [ 43%] FAILED tests/integ/sagemaker/jumpstart/private_hub/estimator/test_jumpstart_private_hub_estimator.py::test_jumpstart_hub_estimator_with_session

4. [gw1] [ 44%] FAILED tests/integ/sagemaker/jumpstart/private_hub/estimator/test_jumpstart_private_hub_estimator.py::test_jumpstart_hub_estimator

5. [gw58] [ 51%] FAILED tests/integ/sagemaker/remote_function/test_decorator.py::test_decorator_torchrun

6. [gw61] [ 59%] FAILED tests/integ/test_training_compiler.py::test_huggingface_pytorch[4.21.1-1.11.0-ml.p3.2xlarge-1]

7. [gw70] [ 73%] FAILED tests/integ/sagemaker/jumpstart/retrieve_uri/test_transfer_learning.py::test_jumpstart_transfer_learning_retrieve_functions

8. [gw25] [ 74%] FAILED tests/integ/sagemaker/modules/train/test_local_model_trainer.py::test_single_container_local_mode_s3_data

9. [gw90] [ 74%] FAILED tests/integ/test_tf.py::test_deploy_with_input_handlers[ml.p3.2xlarge-2.19.0-2.19.0]

@zhaoqizqwang zhaoqizqwang merged commit 0747156 into aws:master-v2 Apr 29, 2026
8 of 11 checks passed
get_kwargs = {"Bucket": bucket, "Key": key}
expected_owner = self.predictor_async.sagemaker_session._get_account_id_if_default_bucket(bucket)
if expected_owner:
get_kwargs["ExpectedBucketOwner"] = expected_owner
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This pattern of passing kwargs is repeated throughout the PR.
Is this the best way to do this?

Can we just pass ExpectedBucketOwner = None to the s3 client if the _get_account_id_if_default_bucket returns None?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previous way of calling the s3 client looked way cleaner IMO.

Comment thread src/sagemaker/session.py

# Explicitly passing a None kms_key to boto3 throws a validation error.
s3_object = s3.get_object(Bucket=bucket, Key=key_prefix)
s3_object = s3.get_object(**get_kwargs)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same pattern of kwargs repeated throughout the codebase. What does documentation say about this and possible other options?

Comment thread src/sagemaker/session.py
# rather than block the S3 operation.
LOGGER.warning(
"Could not resolve caller account id for ExpectedBucketOwner check "
"on bucket %s; proceeding without the check.",
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we swallowing the exception here? Are we ok with the check failing silently in this case?

lucasjia-aws added a commit to lucasjia-aws/sagemaker-python-sdk-lucas that referenced this pull request May 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants