Performance Issue When Retrieving Trie Node

[ajlopez]

In TrieNodeImpl there is this code:

    @Override
    public Optional<Trie> retrieve(byte[] hash) {
        byte[] message = this.store.get(hash);
        if (message == null) {
            return Optional.empty();
        }

        Trie trie = Trie.fromMessage(message, this);
        savedTries.add(trie);
        return Optional.of(trie);
    }

The issue is the line:

        savedTries.add(trie);

Adding the trie node to the set savedTries IMPLIES to execute Trie.getHash() that involves a hash calculation that it is not cheap.

Ie exporting testnet state at height 1784654 executing:

co.rsk.cli.tools.ExportState 1784654 --testnet

WITH the add operation: 150_000 retrieved nodes per hour
WITHOUT the add operation: 180_000 retrieved nodes per hour

Proposal 1

Remove the use of that set, and replace the already save trie node logic by another one that involves less memory and operations

Proposal 2

Inject the hash into trie node in the Trie.fromMessage deserialization logic

There is WIP (Work In Progress) to solve this issue in https://github.com/ajlopez/rskj/tree/triefix