rpc: Implement getblocksbatch

[jamescowens]

Usage:
getblocksbatch <starting block height or hash> <number of blocks> [transaction details]

The return value is JSON object consisting of the number of blocks returned, and the array of blocks. The blocks will include transaction details if that boolean is set to true.

Note that the number of blocks must be between 1 and 1000. Also if the starting height is less than the number of blocks away from the head, the number returned will be limited to the range between the specified starting height and the head.

Note this function is primarily a convenience and performance enhancing function for uses that require high throughput reads of block data, such as the blockchain explorers.

[startailcoon]

Just a quick question. Would it be faster to go on block hash as starting point instead of block number? I know getblockbynumber is slow since it has to find the block in the hash table. I can work with hashes to for starting block

[jamescowens]

I instrumented this with MilliTimer...

Here are the timings for the worst case scenario...
getblocksbatch 0 500 true (i.e. start from genesis).

2021-06-29T20:03:44Z timer getblocksbatch: Finished parsing parameters: elapsed time: 0 ms, time since last check: 0 ms.
2021-06-29T20:03:44Z timer getblocksbatch: Finished finding starting block: elapsed time: 144 ms, time since last check: 144 ms.
2021-06-29T20:03:44Z timer getblocksbatch: Finished populating result for block batch: elapsed time: 516 ms, time since last check: 372

getblocksbatch 1600000 500 true

2021-06-29T20:06:19Z timer getblocksbatch: Finished parsing parameters: elapsed time: 0 ms, time since last check: 0 ms.
2021-06-29T20:06:19Z timer getblocksbatch: Finished finding starting block: elapsed time: 1 ms, time since last check: 1 ms.
2021-06-29T20:06:19Z timer getblocksbatch: Finished populating result for block batch: elapsed time: 269 ms, time since last check: 268
ms.
2021-06-29T20:06:19Z RPCTime : Command getblocksbatch -> Totaltime 275ms

So the middle timing (the loop backwards through pblockindex (pointers) is worst case of 144 ms for all the way back to genesis, and essentially zero at or near the head, and this would be linear. The average is therefore mathematically 1/2 of the worst, or about 72 ms.

You would save 72 ms per call to feed it the hash instead, but it if takes you more than 72 ms to get the hash, you would actually be losing time.

[jamescowens]

We could use a mode flag to be able to either specify a blockhash or blocknumber. This would be a little more complicated to implement, but not much.

[RoboticMind]

Maybe could have a second RPC command that only uses block hashes if that's easier or is the complexity in the hashes themselves?

Also out of curiosity, is there a technical reason for the 500 block limit (e.g minimal time savings) or is it to stop people from accidentally making a really long RPC call or something else?

[jamescowens]

@RoboticMind read my comments to @div72 in the resolved section above.

[jamescowens]

The hashes are very easy to deal with. There is no reason to do two separate commands on this.

[jamescowens]

Ok. I just enhanced this so it can take an overloaded first parameter. You can either specify a block number or a valid hash, and it will figure it out. I also changed the batch size limit to 1000.

[jamescowens]

I will squash all of these commits once we get done fiddling around with this one... :)

[div72]

ACK afad60e.