[WIP] - Creates cached state for SubTreeRoots in darksidewalletd

[pacu]

(When completed) Closes #464

I'm following the same reasoning under GetTreeState, let darksidewalletd provide RPC methods
for clients to stage SubTreeRoots when needing to create a spend before sync test scenario
provides:

• add single root for any shielded protocol
• add many by a streaming call
• clear All
• get a subtree root for a given shielded protocol and height

Please ensure this checklist is followed for any pull requests for this repo. This checklist must be checked by both the PR creator and by anyone who reviews the PR.

• Relevant documentation for this PR has to be completed before the PR can be merged
• A test plan for the PR must be documented in the PR notes and included in the test plan for the next regular release

As a note, all CI tests need to be passing and all appropriate code reviews need to be done before this PR can be merged

[LarryRuane]

I had been implementing a solution to #464, apologies for not mentioning this before you took up this work. My PR is #469, please review that.

A couple of comments on this PR, the description above says that the subtree roots are returned for a given block height, but they're returned for a given index. Secondly, it doesn't look like you're returning the block hash with each root (probably because z_getsubtreesbyindex doesn't return it); this is necessary. For example, on mainnet (production):

$ grpcurl -plaintext -d '{"startIndex": 0, "shieldedProtocol": 0, "maxEntries": 2}' localhost:9067 cash.z.wallet.sdk.rpc.CompactTxStreamer/GetSubtreeRoots
{
  "rootHash": "dUu1k+pC0jGn3fNnZA8Ju/WdwA8sHSADzDQODAFrWxM=",
  "completingBlockHash": "AAAAAACPmsBjbYHceVyw9Ulab9B06BWhnhmfDaSRUf4=",
  "completingBlockHeight": "558822"
}
{
  "rootHash": "A2VMPqy7m5PhIs9td7YG6uKWEPTzikd5hTaBl/1o4C0=",
  "completingBlockHash": "AAAAAAIFzQaAI1LPzYkhgVvQqdc+ta/D7AnzXHGARqk=",
  "completingBlockHeight": "670209"
}

[pacu]

I had been implementing a solution to #464, apologies for not mentioning this before you took up this work. My PR is #469, please review that.

A couple of comments on this PR, the description above says that the subtree roots are returned for a given block height, but they're returned for a given index. Secondly, it doesn't look like you're returning the block hash with each root (probably because z_getsubtreesbyindex doesn't return it); this is necessary. For example, on mainnet (production):

$ grpcurl -plaintext -d '{"startIndex": 0, "shieldedProtocol": 0, "maxEntries": 2}' localhost:9067 cash.z.wallet.sdk.rpc.CompactTxStreamer/GetSubtreeRoots
{
  "rootHash": "dUu1k+pC0jGn3fNnZA8Ju/WdwA8sHSADzDQODAFrWxM=",
  "completingBlockHash": "AAAAAACPmsBjbYHceVyw9Ulab9B06BWhnhmfDaSRUf4=",
  "completingBlockHeight": "558822"
}
{
  "rootHash": "A2VMPqy7m5PhIs9td7YG6uKWEPTzikd5hTaBl/1o4C0=",
  "completingBlockHash": "AAAAAAIFzQaAI1LPzYkhgVvQqdc+ta/D7AnzXHGARqk=",
  "completingBlockHeight": "670209"
}

Oh! thanks for catching that Larry!

[pacu]

Secondly, it doesn't look like you're returning the block hash with each root (probably because z_getsubtreesbyindex doesn't return it); this is necessary. For example, on mainnet (production):

@LarryRuane I mimicked the models in the GRPC proto files. So the blockhash should be returned

type DarksideSubtree struct {
	RootHash              []byte
	CompletingBlockHash   []byte
	CompletingBlockHeight uint64
}

This will mean that the provided tree states will have to contain the block hash. Should that be ok?

[LarryRuane]

Well, in production mode, the block hash is returned and was considered an important part of the interface. So I would think that in darkside mode, it should be provided so that we're fully simulating production mode.

I was hoping you would agree to close this PR and just go with #469. (But the interface is slightly different, so maybe it's not what you want or need.)

[pacu]

I was hoping you would agree to close this PR and just go with #469. (But the interface is slightly different, so maybe it's not what you want or need.)

Idk how I missed that part of the post. Apologize not reviewing yours 🙏

I think this works out. I'm currently updating the Advanced Re Org Tests Regtest datasets to have this new info.

This is closed in favor of #469