Replay slows down

[skzap]

Doing a full replay from scratch starts off pretty fast but slows down over time because of poor memory management. Restarting the node periodically fixes the issue but it is dirty. Avalon needs to more properly manage memory during a replay.

[skzap]

https://github.com/dtube/avalon/tree/memory-fix

[skzap]

I've done some changes on this new branch and will run some tests now. Trying to replay with old code and new code and calculate performance increase

[skzap]

Wasn't that. Still similar speeds.

[skzap]

I think the slowdown issue is the cache.js file now. Skipping the cache backup/rollback system during transaction verification improves the replay speed by a lot and it doesn't seem to slow down. Alas this would break the state of the node on any wrong transaction or block received (let's say a forked node sent a bad block) ...

[skzap]

Ok I did some good progress, replay is much faster now, feel free to try the new memory-fix branch with git checkout memory-fix (just don't do it on your active node :D)

[skzap]

Ok I merged the branch into master. I believe replay speed can still be upgraded by 'clusterizing' the secp256k1 and maybe the sha256s in order to make full usage of multiple cpu cores.