HDDS-5052. [SCM HA Security] Handle leader changes between SCMInfo and getSCMSigned Cert in OM/SCM.

[bharatviswa504]

What changes were proposed in this pull request?

This Jira is to handle leader change between getScmInfo and getScmSignedCert.

Problem:
Leader is SCM1 - Returned SCMID is SCM1ID
ScmInfo returns the leader SCMID which is SCM1ID.

Leader is SCM2 - SCM ID is SCM2ID
getSCMSignedCert, during generate certificate it has a check compare the scmId passed in CSR, is same as current SCM scmID

In this case when the leader change between these 2 calls OM will fail to get a Certificate.

Proposed Solution:
Discuss with Xiaoyu offline, and we decided to eliminate persist scmId and check SCMID during OM startup.

We persist SCM ClusteriD for both HA/non-HA in OzoneManager.

And in SCM we remove the SCMID check and keep the SCM ClusterID check. (ClusterID is unique for a single Ozone cluster)

In SCM previously sub-CA subject looks like CN=scm-sub@host,OU=fetchedSCMID from primary,O=ClusterID
After this change sub-CA subject will be CN=scm-sub@host,OU=local SCMID,O=ClusterID

For OM/DN the subject names are the same as before.

And as SCMID is used in Subject CN name, not touched that, so that the certificate subject will be in the same format before/after the upgrade.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-5052

How was this patch tested?

Existing docker tests.

[bharatviswa504]

The last commit is the changes proposed in this PR. It is dependent on HDDS-4915.

[bharatviswa504]

Rebased now. It is ready for review.

[bshashikant]

Looks good to me.

[bharatviswa504]

Thank You @bshashikant for the review.