Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-27042. Remove S3Guard awareness from HBoss #33

Closed

Conversation

steveloughran
Copy link
Contributor

  • Cut HBoss test code which would explicitly disable S3Guard
    -obsolete and breaks the build on forthcoming hadoop releases.
  • Also cut all references in docs and XML.
  • EmbeddedS3 to implement deleteObject() and so support
    AWS SDKs 1.11.1026+, which expects this (hadoop 3.3.2+).
  • Maven test runs to provide full stack traces on failures,
    so runtime failures can be debugged and fixed.

* Cut HBoss test code which would explicitly disable S3Guard
  -obsolete and breaks the build on forthcoming hadoop releases.
* Also cut all references in docs and XML.
* EmbeddedS3 to implement deleteObject() and so support
  AWS SDKs 1.11.1026+, which expects this (hadoop 3.3.2+).
* Maven test runs to provide full stack traces on failures,
  so runtime failures can be debugged and fixed.

Change-Id: Iab10cfc6780cea064ea116efc1c177332d780514
@steveloughran
Copy link
Contributor Author

tested in the mock client, and in progress is a test with fs.hboss.data.uri pointing at a real s3a bucket. not convinced that this setting is being picked up though.

@steveloughran
Copy link
Contributor Author

(yes, it was being picked up, adding an invalid bucket triggered test run). bulild options were

-Phadoop3.3 -Dhadoop33.version=3.3.3 

@steveloughran
Copy link
Contributor Author

s3 london; directory markers=keep

Tests run: 225, Failures: 0, Errors: 0, Skipped: 55

Copy link
Member

@joshelser joshelser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for updating the documentation at the same time, Steve!

@joshelser
Copy link
Member

joshelser commented May 23, 2022

Running the tests locally since we don't have QA wired up here automatically.

Edit - All of these passed:

 5026  mvn package -Dhadoop.profile=3.2
 5027  mvn package -Dhadoop.profile=3.3
 5028  mvn package -Dhadoop.profile=3.3 -Dhadoop33.version=3.3.3

@joshelser joshelser closed this May 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants