Reducing the number of blocks that Raiden client needs to read on startup

[pirapira]

Currently, for figuring out the current state of a TokenNetwork, Raiden client reads all blocks since the contract's deployment. This issue keeps track of designing and implementing a way to reduce the number of blocks that Raiden clients need to read when they start up.

[pirapira]

I currently do not know how to achieve this. I guess the contract can provide a hash of the global state, and other offchain parties can provide the corresponding global state. Now how does this global state look like? @hackaugusto

[hackaugusto]

The token network does not save all data in storage but publish it with events, for a node to recover the network state it must fetch all events since the smart contract deployment block. This is a problem for two reasons:

• The node must fetch all events and apply them in order since the smart contract was deployed. This means the number of blocks will increase linearly with time.
• Raiden cannot be used with clients that trim events.

I can think of two possible solutions for the above:

• Remove the optimization, and don't discard the data. This, however, would increase the cost of using the smart contract which is undesirable.
• Have a mechanism to store the data off-chain, which can be validated against on-chain data.

For the second approach, here are a few ideas:

• Propose a new JSON RPC interface, eth_getStorageHash which would return the hash of the smart contract storage at a given block height. If that is available then we can provide the smart contract data by an off-band medium (from the blockchain perspective), this can be done by a service like the PFS/MS/SpecializedStorageService or a distributed storage. The data would be split into two halves, smartcontract data to reconstruct the storage hash, and the necessary data which is not part of the contract's storage but describes the network and can be validate against it.
• Without the above, another strategy is to compute and maintain a hash representing the smart contract state ourselves. The simplest approach is to maintain a single storage slot with the aggregated history of the smart contract operations, at the end of each successful function call, the smart contract would update the hash as follows current_hash = hash(current_hash || function_selector || function_arguments). This does work, however, we have additional on-chain computation and storage requirements, and the off-chain state would increase linearly with the number of operations in the smart contract, which is better then the fetching of all events, it works but it is not ideal.
• A third option is a poor man's version of the eth_getStorageHash, using the existing eth_getStorageAt and a service. In this case the provided data would have the same two halves, however the smart contract half would necessarily have every single storage address, and the node would have to do an RPC call for each storage address to validate the data, instead of just computing the hash in a deterministic manner. This has the downside of having a linear number of RPC calls, proportional to the current used storage, and we must add very specific code to validate the data, to make sure the service is not omitting data. This is not ideal but can be implemented right now and should be an improved against the event fetching.

I have the impression that any alternative to improve the second option will require additional storage, which is just silly and I would then prefer to remove the optimization and just have the data available.

[hackaugusto]

Here is a third approach. We can keep the storage optimizations, but remove the need for the node to poll for all the events since the smart contract was deployed.

This can be achieved by making the channel open an operation that needs both parties (e.g with Add off-chain settlement period negotiation raiden-network/raiden#864). If we add an expiration to the above, both nodes will know that a channel will be created, so they can start polling for the event, if the negotiation expires both nodes can stop the polling (If the channel is created than a full node has to keep polling, to know if the channel was closed). I would forbid two channels from being open with the same agreement, this is already implemented since ATM only one channel among two parties is allowed.

Edit: This would only help if the internal routing is removed from the python client, Remove the local routing module raiden-network/raiden#4982