Merge HDDS-3698-nonrolling-upgrade branch into master#2390
Merged
avijayanhwx merged 97 commits intomasterfrom Jul 12, 2021
Merged
Merge HDDS-3698-nonrolling-upgrade branch into master#2390avijayanhwx merged 97 commits intomasterfrom
avijayanhwx merged 97 commits intomasterfrom
Conversation
…s with lower MLV to be used in pipelines. (#2204)
…ing-upgrade-master-merge2 * upstream/master: (56 commits) HDDS-2212. Genconf tool should generate config files for secure clust… (#1788) HDDS-5166. Remove duplicate assignment of OZONE_OPTS for freon and sh (#2195) Revert "HDDS-5144. Create github check to alert when dependency tree is changed (#2177)" HDDS-4983. Display key offset for each block in command key info (#2051) HDDS-5144. Create github check to alert when dependency tree is changed (#2177) HDDS-4585. Support bucket acl operation in S3g (#1701) HDDS-5153. Decommissioning a dead node should complete immediately (#2190) HDDS-5147. Intermittent test failure in TestContainerDeletionChoosingPolicy#testRandomChoosingPolicy (#2188) HDDS-5152. Fix Suggested leader in Client. (#2189) HDDS-5148. Bump ratis version to 2.1.0-ff8aa66-SNAPSHOT (#2184) HDDS-4515. Datanodes should be able to persist and load CRL (#2181) HDDS-5060. [SCM HA Security] Make InterSCM grpc channel secure. (#2187) HDDS-5051. Ensure failover to suggested leader if any for NotLeaderException. (#2141) HDDS-5127. Fix getServiceList when SCM HA is enabled (#2173) HDDS-4889. Add simple CI check for docs (#2156) HDDS-5131. Use timeout in github actions (#2176) HDDS-5103. Fix Install Snapshot Mechanism in SCMStateMachine. (#2155) HDDS-5124. Use OzoneConsts.OZONE_TIME_ZONE instead of "GMT" (#2166) HDDS-5047. Refactor Pipeline to use ReplicationConfig instead of factor/type (#2096) HDDS-5083. Bump version of common-compress (#2139) ... Conflicts: hadoop-hdds/common/pom.xml hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/HddsDatanodeService.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineManager.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMStorageConfig.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneClusterImpl.java hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMStorage.java hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconStorageContainerManagerFacade.java
Update javassist in recon pom Fix changes introduced in merge that failed TestSCMNodeManager upgrade tests Fix checkstyle Fix intermittent test failure TestSCMNodeManager#testSetNodeOpStateAndCommandFired after merge Skip scm init default layout version in TestOzoneConfigurationFields
…lready being used. (#2257)
… layout version. (#2265)
…ing-upgrade-master-merge * upstream/master: (76 commits) HDDS-5280. Make XceiverClientManager creation when necessary in ContainerOperationClient (#2289) HDDS-5272. Make ozonefs.robot execution repeatable (#2280) HDDS-5123. Use the pre-created apache/ozone-testkrb5 image during secure acceptance tests (#2165) HDDS-4993. Add guardrail for reserved buffer size when DN reads a chunk (#2058) HDDS-4936. Change ozone groupId from org.apache.hadoop to org.apache.ozone (#2018) HDDS-4043. allow deletion from Trash directory without -skipTrash option (#2110) HDDS-4927. Determine over and under utilized datanodes in Container Balancer. (#2230) HDDS-5273. Handle unsecure cluster convert to secure cluster for SCM. (#2281) HDDS-5158. Add documentation for SCM HA Security. (#2205) HDDS-5275. Datanode Report Publisher publishes one extra report after DN shutdown (#2283) HDDS-5241. SCM UI should have leader/follower and Primordial SCM information (#2260) HDDS-5219. Limit number of bad volumes by dfs.datanode.failed.volumes.tolerated. (#2243) HDDS-5252. PipelinePlacementPolicy filter out datanodes with not enough space. (#2271) HDDS-5191. Increase default pvc storage size (#2219) HDDS-5073. Use ReplicationConfig on client side (#2136) HDDS-5250. Build integration tests with Maven cache (#2269) HDDS-5236. Require block token for more operations (#2254) HDDS-5266 Misspelt words in S3MultipartUploadCommitPartRequest.java line 202 (#2279) HDDS-5249. Race Condition between Full and Incremental Container Reports (#2268) HDDS-5142. Make generic streaming client/service for container re-replication, data read, scm/om snapshot download (#2256) ... Conflicts: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocol.java hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/protocolPB/StorageContainerLocationProtocolClientSideTranslatorPB.java hadoop-hdds/interface-admin/src/main/proto/ScmAdminProtocol.proto hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/MockNodeManager.java hadoop-ozone/dist/src/main/compose/testlib.sh hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestStorageContainerManager.java hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto hadoop-ozone/ozone-manager/pom.xml hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/utils/OzoneManagerRatisUtils.java hadoop-ozone/s3gateway/pom.xml
Update jar report Fix failing SCMNodeManager upgrade tests after merge Fix reflections loading upgrade actions Works around reflections bug ronmamo/reflections#178 Which was causing mock actions for testing to be registered in non-test code. Correctly fix jar-report Downgrade reflections to 0.9.11 Disable fail fast for testing Wait for ratis 3 pipeline to be open in post scm finalize Fix javassist shadowing that was breaking startup Revert "Disable fail fast for testing" This reverts commit c048c0c.
|
SonarCloud Quality Gate failed.
|
swagle
approved these changes
Jul 8, 2021
Contributor
swagle
left a comment
swagle
left a comment
There was a problem hiding this comment.
+1 for the branch merge PR
Contributor
|
It seems like the acceptance test failure is unrelated. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Merge non-rolling upgrade feature into master. See Jira for design documents and details.
What is the link to the Apache JIRA
HDDS-3698
How was this patch tested?
Unit, integration, and acceptance tests added and updated. Manual testing done as well.