feat(zebra-scan): Create a scanner results channel

[oxarbitrage]

This task is about creating a results channel within the zebra-scan crate. The results channel is designed to transmit related transactions for all stored keys over the channel.

To accomplish this task, we need to:

• Define the structure of the channel, specifying that the data in the channel should consist of triples in the form of (AccountId [linked to key], block hash/height, Vec<WalletTx>).
• Create the channel when the module is initialized using zebra-scan::new() or similar.

If we don't want to drop any results, we should use a mpsc channel for results.

Out of Scope

If we use a tokio, futures, or std channel, we don't need to do these tests, because those channels are already well-tested:

• Implement the results channel functionality to ensure that related transactions are properly transmitted.
• Create tests to verify the correct functioning of this channel.

The output of the scanner task should send data to this channel but this will be done in another ticket.

In a later ticket we can send the results directly to storage, and read results out of storage.

Account filtering - out of scope

Use a separate API to filter results per account.

Have a separate channel and results client for each account:

• Store an mpsc channel for every client (HashMap<Client, Vec<(SaplingIvk, RangeBounds<Height>)>>)
• Send transactions that are trial decrypted by a client's viewing keys directly to that client

[teor2345]

@arya2 @oxarbitrage I think there's a design decision we need to make here:

Is it ok for all broadcast channel receivers to receive all the results of the scanner task?
Or do we need to split results by the account or viewing key inside the task?

If we send all results to all receivers, we can filter or split them later if we need to. I'd like to do that outside the scanner task to reduce the complexity of the task, and the assumptions we're making about the rest of the design. (If we assume too much here, we'll have to rewrite the code later if the design changes.)

For example, the broadcast channel receiver could store results in the scanner storage. Then the result reading API could check the account ID in the channel or storage before returning results for that ID.

[teor2345]

I changed the title of the ticket because a broadcast channel is a specific type of tokio channel, but we haven't decided to use that type of channel yet.

[teor2345]

• If we don't want to drop any results, use a mpsc channel instead, send the results directly to storage, and read results out of storage

It looks like we'll have a scanner task for new incoming blocks, and multiple tasks for cached state blocks (each new key needs to re-scan the whole cached state). So a mpsc channel to storage seems like a good design here.

#7908 (comment)

[arya2]

Is it ok for all broadcast channel receivers to receive all the results of the scanner task?
Or do we need to split results by the account or viewing key inside the task?

I feel it would be simpler to split the results by account or client.

read results out of storage

We could also send unordered relevant transactions and let clients make a new request with a set of queries that fill the gaps when Zebra restarts.

[mpguerra]

Hey team! Please add your planning poker estimate with Zenhub @arya2 @oxarbitrage @teor2345 @upbqdn

[teor2345]

Is it ok for all broadcast channel receivers to receive all the results of the scanner task?
Or do we need to split results by the account or viewing key inside the task?

I feel it would be simpler to split the results by account or client.

read results out of storage

We could also send unordered relevant transactions and let clients make a new request with a set of queries that fill the gaps when Zebra restarts.

I think it will be simpler to read the results out of storage, and leave the filtering by account or key to the caller of the storage reads.

[teor2345]

This ticket might not be needed, writing results directly to the database seems to work just fine for now.