HDDS-7831. Use symmetric secret key to sign and verify token

[duongkame]

What changes were proposed in this pull request?

Use symmetric secret keys to sign and verify (container and block) tokens.

This implements the token signing and verifying following Proposal #2 (secret key pulling model).

The detailed structure of the components looks like below:

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-7831

How was this patch tested?

• Existing integration and acceptance tests that verify basic end-to-end workflow.
• Specific edge cases (e.g. block access at the rotation boundary) will be implemented as per HDDS-8003.
• A performance evaluation of this implementation has been done as per HDDS-8143.

[fapifta]

Hi @duongkame, thank you for working on the patch, overall it looks good, but please find a few inline comments below that I would like to either discuss or to be addressed before we get this in.

[fapifta]

I was wondering if we really need both of these constructor... I understand that we have two from the older version as well, but the only difference is how we get the blockId parameter, as a BlockID or as a String.
The String representation is used in the protocol, I believe we should not use it elsewhere, but apparently we are doing it right now. I think this we might address along with the series of changes that we are doing to go from asymmetric to symmetric keys. What do you think?
I don't want to address this right now, but should we go ahead and have a task for this?

[duongkame]

Got the concern. I think the internal logic of OzoneBlockTokenId always favor a string presentation. The BlockId if passed is converted to a "service id".

  public static String getTokenService(BlockID blockID) {
    return String.valueOf(blockID.getContainerBlockID());
  }

and this "service id" is used to verify the token .

    // check token service (blockID or containerID)
    String service = String.valueOf(getService(cmd));
    if (!Objects.equals(service, tokenId.getService())) {
      throw new BlockTokenException("ID mismatch. Token for ID: " +
          tokenId.getService() + " can't be used to access: " + service +
          " by user: " + tokenUser);
    }

[fapifta]

Before this code piece, there is the regular checks for certificate lifetime, and existence. Do we still need those?

[duongkame]

I think the check you mention is in the DefaultCertificateClient, and it's independent from the implementation of ContainerTokenSecretManager.

[ChenSammi]

@duongkame , thanks for working on this. It's really quite a big patch. Can we separate it into small ones? It's hard to find the exact piece of code line to add comments.

[duongkame]

Hi @duongkame, thank you for working on the patch, overall it looks good, but please find a few inline comments below that I would like to either discuss or to be addressed before we get this in.

Thanks for the helpful review @fapifta. I've looked at the inline comments.

[duongkame]

@duongkame , thanks for working on this. It's really quite a big patch. Can we separate it into small ones? It's hard to find the exact piece of code line to add comments.

Thanks for looking at this @ChenSammi. At the beginning I tried to deliver in smaller pieces, e.g. separate token generator and verifier, or block and container tokens... But I couldn't figure out a good way to do that because the logic parts are very coherent. E.g. the token verifier depends on the generator, block token and container token share the same root...

Anyway, the number of changed files are huge because of many adapted test cases. The number of core changes are limited in the diagram in the PR description.

[umamaheswararao]

Yeah it's a large patch. But looks like it may be hard to separate as small pieces as lot of the files touched due to passing over the variables etc. Just wondering whether we can just provide implementation for some of the new classes like SecretKey*Client etc as this is in branch. That may not reduce much I guess. Please take care of avoid touching with some unrelated changes here as this patch is already big ex: OzoneSecurityUtil.java

[umamaheswararao]

@fapifta and @ChenSammi do you have further comments on this?

[kerneltime]

I went over this PR once more, there are clean up tasks that need to be done and renames for classes, mergint this in as the change looks good and there will be follow up tasks for this project.