Add merging to BlockRange

[nick-mobilecoin]

No description provided.

[github-actions]

⚠️ Downstream repo mobilecoinofficial/full-service failed to build. Check actions status for details.

[github-actions]

⚠️ Downstream repo mobilecoinofficial/android-bindings failed to build. Check actions status for details.

[github-actions]

⚠️ Downstream repo mobilecoinofficial/android-bindings failed to build. Check actions status for details.

[github-actions]

⚠️ Downstream repo mobilecoinofficial/full-service failed to build. Check actions status for details.

[nick-mobilecoin]

from internal convo

I’m curious:

1. how you imagine using index
2. what utility there is in creating a merged_range with merged_range.start_block != 0

1. The index was added to support determining how to populate the CheckedKeyImageResponse,

   mobilecoin/fog/ledger/enclave/impl/src/lib.rs

   Line 262 in 617786e

   let num_blocks = shard_query_responses

   . My hope is that with the index the caller will then be able to copy that response directly from the shard that provided the "final" range that got merged. I'm up for ideas on this one. I think we need to ensure that the caller doesn't accidentally provide a larger global txo count than the number of blocks. Lets assume there are 3 txos per block and the ranges are (0,10), (10, 12), (20, 22). The intent is that our max block returned is 12, I'm thinking we also need to ensure the gobal_txo_count is 36 and not 66.

2. I don't think there is any use in providing a merged range that doesn't start at 0, for the system. However for this function I think it's already complicated enough with the index piece. My thought was that the caller would do something like (non compiling code)

let zero_range = BlockRange::new(0,1);
let received_ranges = ranges.sort(); // some logic somewhere that keeps the indices when sorted.
let (index, merged_range) = BlockRange.merge_from_ranges([&zero_range].chain(received_ranges));

Up for suggestions as I recognize this may be placing too much room for error by the callers

[holtzman]

from internal convo

I’m curious:

1. how you imagine using index
2. what utility there is in creating a merged_range with merged_range.start_block != 0

1. The index was added to support determining how to populate the CheckedKeyImageResponse,

   mobilecoin/fog/ledger/enclave/impl/src/lib.rs

   Line 262 in 617786e

   let num_blocks = shard_query_responses

   
   . My hope is that with the index the caller will then be able to copy that response directly from the shard that provided the "final" range that got merged. I'm up for ideas on this one. I think we need to ensure that the caller doesn't accidentally provide a larger global txo count than the number of blocks. Lets assume there are 3 txos per block and the ranges are (0,10), (10, 12), (20, 22). The intent is that our max block returned is 12, I'm thinking we also need to ensure the gobal_txo_count is 36 and not 66.
2. I don't think there is any use in providing a merged range that doesn't start at 0, for the system. However for this function I think it's already complicated enough with the index piece. My thought was that the caller would do something like (non compiling code)

let zero_range = BlockRange::new(0,1);
let received_ranges = ranges.sort(); // some logic somewhere that keeps the indices when sorted.
let (index, merged_range) = BlockRange.merge_from_ranges([&zero_range].chain(received_ranges));

Up for suggestions as I recognize this may be placing too much room for error by the callers

Ah. Fessing up to initiating the internal conversation, and this makes sense to me.

[holtzman]

Up for suggestions as I recognize this may be placing too much room for error by the callers

What if each BlockRange included the query_response from the corresponding shard? Then instead of doing a merge_from_range, do abest_query_response where all of the shard query_responses are sorted by start_block and looked at in order until a discontinuity is found (or the end of the sorted BlockRange list is reached) and the query_response from before the discontinuity (or final one in list if there is no discontinuity) is returned?

Then num_blocks, global_txo_count, and latest_block_version could be taken from that query_response.

Some logic would also be needed in case none of the shards reported results that included block 0.

[nick-mobilecoin]

Up for suggestions as I recognize this may be placing too much room for error by the callers

What if each BlockRange included the query_response from the corresponding shard? Then instead of doing a merge_from_range, do abest_query_response where all of the shard query_responses are sorted by start_block and looked at in order until a discontinuity is found (or the end of the sorted BlockRange list is reached) and the query_response from before the discontinuity (or final one in list if there is no discontinuity) is returned?

Then num_blocks, global_txo_count, and latest_block_version could be taken from that query_response.

Some logic would also be needed in case none of the shards reported results that included block 0.

This may be a better alternative. The shards are going to need to return a different type then the CheckKeyImagesResponse in order to accomodate the range. So it may make sense to create a dedicated NewResponseType::merge_from(responses) and contain the logic there.

I'll keep this PR open/unmerged for now and build the other logic on top of this to see how it plays out.

[nick-mobilecoin]

Follow on PR is #3933

[github-actions]

⚠️ Downstream repo mobilecoinofficial/android-bindings failed to build. Check actions status for details.

[github-actions]

⚠️ Downstream repo mobilecoinofficial/full-service failed to build. Check actions status for details.

[github-actions]

⚠️ Downstream repo mobilecoinofficial/android-bindings failed to build. Check actions status for details.

[github-actions]

⚠️ Downstream repo mobilecoinofficial/full-service failed to build. Check actions status for details.