HDDS-6303. Merge from master to resolve CI workflow issues & hrpcOmTransport support #3074
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…he#2981) * HDDS-6175. Use s3Auth during proxy during decrypt in RpcClient.
…teServiceWithFailingSCM. (apache#2991)
…stOMVolumeSetQuotaRequest. (apache#2982)
* HDDS-6125. Recon get limited delta updates from OM * HDDS-6215. Fix unit test * trigger new CI check * HDDS-6215. Fix typo * trigger new CI check Co-authored-by: Symious <yiyang0203@gmail.com>
…he#3063) Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>
…ch green builds. Merging with Master branch impacts HDDS-4440 PRs in progress. This merge also requires partial changes that were initially in PR for HDDS-5545 to be included. Namely, the patch to allow switching between OmTransports, GrpcOmTransport and Hadoop3OmTransport through ozone configuration in the OmConfigKeys.
…nsport for s3 gateway - TestOzoneConfigurationFields (added config key not in xml).
adoroszlai
added a commit
to adoroszlai/ozone
that referenced
this pull request
Apr 11, 2022
…hrpcOmTransport support (apache#3074)" This reverts commit b88a45e.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Necessary due to CI workflow issue, HDDS-6239, impacting feature branch green builds. This merge also requires partial changes that were initially in PR for HDDS-5545 to be included. Namely, the patch to allow switching between OmTransports,
GrpcOmTransportandHadoop3OmTransportthrough ozone configuration in theOmConfigKeys.The current merge from the Master includes changes to interfaces for the s3 gateway client producer, through the
OzoneClientCache, that removes the need for the omServiceId instead it uses the OmUtils to get the omServiceId through the ozone configuration file. The master merge also includes implementation and tests for handling ozone federation configurations with multiple omServiceIds. In the merged master, unit tests for the s3 gateway were added to test multiple omServiceIds. These new unit tests require changes to the current s3g feature branch that were included in the PR for HDDS-5545. For this those changes that allowed switching omTransports through configuration for the s3 gateway are put into this PR, thereby splitting the PR for 5545. These patches applied to this merge passes all s3 gateway unit tests. These patches include:Changes to the
OmTransportFactory.java(switch between service provider and configuration OmTransports),OmConfigKeys.java(set the OmTransport to be used by the s3 gateway),and the unit test
TestGrpcOmTransport.java.With this patch, by default the s3 gateway uses hadoop rpc and the Hadoop3OmTransport connection between the s3g and the om. The configuration variable affecting this is
ozone.om.transport.classset toorg.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactoryfor hrpc (default) and set toorg.apache.hadoop.ozone.om.protocolPB.GrpcOmTransportFactoryfor s3g Grpc connection.What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-6303
How was this patch tested?
S3 gateway unit tests, PASSED all tests.
MR acceptance tests - PASSED