Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HDDS-5273. Handle unsecure cluster convert to secure cluster for SCM. #2281

Merged
merged 1 commit into from May 27, 2021
Merged

HDDS-5273. Handle unsecure cluster convert to secure cluster for SCM. #2281

merged 1 commit into from May 27, 2021

Conversation

bharatviswa504
Copy link
Contributor

@bharatviswa504 bharatviswa504 commented May 25, 2021 •
edited

What changes were proposed in this pull request?

In SCM sub-ca certs are set up during init, if a cluster is converted to secure later, in else part of the scmInit, we need to initialize security.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-5273

How was this patch tested?

I Will test out the fix on the cluster and update here. Posting fix for CI run.

Tested this on a cluster and later it is converted to secure.

2021-05-26 01:15:23,478 INFO org.apache.hadoop.hdds.scm.ha.HASecurityUtils: Init response: GETCERT
2021-05-26 01:15:23,815 INFO org.apache.hadoop.hdds.scm.ha.HASecurityUtils: Successfully stored SCM signed certificate.
2021-05-26 01:15:23,822 INFO org.apache.hadoop.hdds.scm.server.StorageContainerManager: SCM already initialized. Reusing existing cluster id for sd=/var/lib/hadoop-ozone/scm/data/scm;cid=CID-9efddf03-b533-4134-816a-a35b87eec46b; layoutVersion=0; HAEnabled=false

xiaoyuyao
xiaoyuyao approved these changes May 26, 2021
View reviewed changes
Copy link
Contributor

@xiaoyuyao xiaoyuyao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bharatviswa504 for working on this. PR LGTM, +1.

@bharatviswa504 bharatviswa504 merged commit d567360 into apache:master May 27, 2021
@bharatviswa504
Copy link
Contributor Author

Thank You @xiaoyuyao for the review.

errose28 added a commit to errose28/ozone that referenced this pull request Jun 1, 2021
@errose28
Merge remote-tracking branch 'upstream/master' into HDDS-3698-nonroll…
1d18f47 
…ing-upgrade-master-merge

* upstream/master: (76 commits)
  HDDS-5280. Make XceiverClientManager creation when necessary in ContainerOperationClient (apache#2289)
  HDDS-5272. Make ozonefs.robot execution repeatable (apache#2280)
  HDDS-5123. Use the pre-created apache/ozone-testkrb5 image during secure acceptance tests (apache#2165)
  HDDS-4993. Add guardrail for reserved buffer size when DN reads a chunk (apache#2058)
  HDDS-4936. Change ozone groupId from org.apache.hadoop to org.apache.ozone (apache#2018)
  HDDS-4043. allow deletion from Trash directory without -skipTrash option (apache#2110)
  HDDS-4927. Determine over and under utilized datanodes in Container Balancer. (apache#2230)
  HDDS-5273. Handle unsecure cluster convert to secure cluster for SCM. (apache#2281)
  HDDS-5158. Add documentation for SCM HA Security. (apache#2205)
  HDDS-5275. Datanode Report Publisher publishes one extra report after DN shutdown (apache#2283)
  HDDS-5241. SCM UI should have leader/follower and Primordial SCM information (apache#2260)
  HDDS-5219. Limit number of bad volumes by dfs.datanode.failed.volumes.tolerated. (apache#2243)
  HDDS-5252. PipelinePlacementPolicy filter out datanodes with not enough space. (apache#2271)
  HDDS-5191. Increase default pvc storage size (apache#2219)
  HDDS-5073. Use ReplicationConfig on client side  (apache#2136)
  HDDS-5250. Build integration tests with Maven cache (apache#2269)
  HDDS-5236. Require block token for more operations (apache#2254)
  HDDS-5266 Misspelt words in S3MultipartUploadCommitPartRequest.java line 202 (apache#2279)
  HDDS-5249. Race Condition between Full and Incremental Container Reports (apache#2268)
  HDDS-5142. Make generic streaming client/service for container re-replication, data read, scm/om snapshot download (apache#2256)
  ...

Conflicts:
	hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocol.java
	hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/protocolPB/StorageContainerLocationProtocolClientSideTranslatorPB.java
	hadoop-hdds/interface-admin/src/main/proto/ScmAdminProtocol.proto
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java
	hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/MockNodeManager.java
	hadoop-ozone/dist/src/main/compose/testlib.sh
	hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/TestStorageContainerManager.java
	hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto
	hadoop-ozone/ozone-manager/pom.xml
	hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
	hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/utils/OzoneManagerRatisUtils.java
	hadoop-ozone/s3gateway/pom.xml
bharatviswa504 added a commit to bharatviswa504/hadoop-ozone that referenced this pull request Jul 25, 2021
@bharatviswa504
HDDS-5273. Handle unsecure cluster convert to secure cluster for SCM. (…
aae6905 
…apache#2281)

Change-Id: I217ffff908a6d47aa9a5189dd6250d5410c4e7b3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xiaoyuyao xiaoyuyao xiaoyuyao approved these changes

@bshashikant bshashikant Awaiting requested review from bshashikant

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@bharatviswa504 @xiaoyuyao