Skip to content

HDDS-16071. Add S3 Bucket Lifecycle CRUD integration tests to AbstractS3SDKV2Tests - #10986

Open
NickJavaDev88 wants to merge 5 commits into
apache:masterfrom
NickJavaDev88:HDDS-16071
Open

HDDS-16071. Add S3 Bucket Lifecycle CRUD integration tests to AbstractS3SDKV2Tests#10986
NickJavaDev88 wants to merge 5 commits into
apache:masterfrom
NickJavaDev88:HDDS-16071

Conversation

@NickJavaDev88

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This PR adds basic S3 Bucket Lifecycle CRUD integration tests to AbstractS3SDKV2Tests to achieve test parity with AbstractS3SDKV1Tests.

Key observations ported for AWS SDK v2:

  • getBucketLifecycleConfiguration throws an S3Exception (404 / NoSuchLifecycleConfiguration) when no lifecycle configuration exists on a bucket (unlike SDK v1 which returned null).
  • deleteBucketLifecycle remains idempotent (204 No Content) when deleting a non-existent configuration, matching the fix from HDDS-16005.
  • Creating an invalid rule fails with InvalidRequest (400).

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16071

How was this patch tested?

  • Static checks (checkstyle.sh, rat.sh, author.sh) passed.
  • Ran integration tests locally: mvn -pl :ozone-integration-test-s3 test -Dtest=TestS3SDK (194 tests passed).
  • CI run: https://github.com/NickJavaDev88/ozone/actions/runs/31397269975
    (Note: The single failure in acceptance is a known flaky test where SCM gets stuck in safe mode, unrelated to S3 SDK changes).

@echonesis echonesis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @NickJavaDev88 for the patch.

() -> s3Client.putBucketLifecycleConfiguration(b -> b
.bucket(nonExistentBucket)
.lifecycleConfiguration(validConfig)));
assertEquals(404, exception2.statusCode());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use HTTP_NOT_FOUND instead of the hard-coded 404, consistent with the existing lifecycle tests.

Comment on lines +2490 to +2491
assertThrows(S3Exception.class,
() -> s3Client.getBucketLifecycleConfiguration(b -> b.bucket(bucketName)));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we assert HTTP_NOT_FOUND and S3ErrorTable.NO_SUCH_LIFECYCLE_CONFIGURATION.getCode(), as mentioned in the PR description? The same applies to the assertions below.

@NickJavaDev88

Copy link
Copy Markdown
Contributor Author

Thanks for the review, @echonesis — both addressed in 3b89d36 :

  • Replaced the hard-coded 404 with HttpURLConnection.HTTP_NOT_FOUND.
  • Every assertThrows(S3Exception.class, ...) in the new lifecycle tests now captures the exception and asserts both statusCode() and awsErrorDetails().errorCode() against the matching S3ErrorTable constant, instead of just checking that something was thrown.

Verified locally: all 4 new lifecycle tests (testS3LifecycleConfigurationCreateSuccessfully, CreationFailed, Delete, Get) pass against a real MiniOzoneCluster in both AbstractS3SDKV1Tests and AbstractS3SDKV2Tests.
CI is running on the latest commit: https://github.com/NickJavaDev88/ozone/actions/runs/31464196500

@priyeshkaratha priyeshkaratha added s3 S3 Gateway s3-lifecycle HDDS-8342 labels Aug 11, 2026
@NickJavaDev88

Copy link
Copy Markdown
Contributor Author

Thanks @priyeshkaratha for adding the labels!

@NickJavaDev88

Copy link
Copy Markdown
Contributor Author

I pushed a quick follow-up commit (c849e67) to replace the remaining literal 400 with HttpURLConnection.HTTP_BAD_REQUEST in testS3LifecycleConfigurationCreationFailed.

I initially missed this one during my search because I was filtering by 404, but caught it while reviewing the test cases where the bucket exists but lacks configuration.

The CI run is currently in progress: https://github.com/NickJavaDev88/ozone/actions/runs/31678412705

@priyeshkaratha

Copy link
Copy Markdown
Member

Can you do rebase with latest master? @NickJavaDev88 There are few changes happened which is improving CI run in master.

@NickJavaDev88

Copy link
Copy Markdown
Contributor Author

Thanks for the heads-up, @priyeshkaratha! I've brought the branch up to date with master.

I updated it via merge as recommended in the CONTRIBUTING.md guide. This pulls in the latest CI improvements, and the new run is available here: https://github.com/NickJavaDev88/ozone/actions/runs/31786388152

@sreejasahithi sreejasahithi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @NickJavaDev88 for this PR.
Changes LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

s3-lifecycle HDDS-8342 s3 S3 Gateway

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants