Rebuild witness and nullifier caches in wallet if a large reorganisation occurs

[str4d]

In #1233 a cache of incremental witnesses per spendable note was added to the wallet, and a corresponding cache of anchors nullifiers. The size of the cache is intended to be large enough that if a chain reorganisation occurs, the cache will not be exhausted. However, in rare cases (e.g. extended network splits), a large reorganisation might occur, and the wallet will cease to behave correctly if the cache is exhausted. In this case, we should rebuild the cache from scratch.

[ebfull]

This can very well be something we don't launch with if the anchor cache in the wallet is large enough.

[str4d]

The anchor cache is currently the same size as the coinbase maturity duration, so if we do encounter this issue, we'd also be dealing with the issue of transactions becoming invalid due to non-existent coinbase.

[nathan-at-least]

Let's defer this until #713 is complete, since that will give us a known boundary condition, and may make this unnecessary.

[str4d]

I don't believe #713 will make this unnecessary. Forward movement on re-organization happens in chunks of at most 32 blocks (to prevent ActivateBestChain from holding the main lock for too long), so it is possible that two re-orgs could occur in succession that individually are below the re-org limit, but together exceed the cache size. This would require three separate chain forks to occur, with the most recent fork parent under 100 blocks ago, and the second fork parent somewhere between 100 and 200 blocks ago. The node would then need to learn of the more recent chain fork first, followed immediately by the older chain fork.

[daira]

#3443 demonstrated that it is possible to fix this problem, but it still isn't entirely fixed. Bumping milestone to 2.0.1.

[arielgabizon]

Ran into this while trying to run the pruning test which does a 288 block reorg #2815 (comment)

[jan-matejka]

Keep running into this zcashd: wallet/wallet.cpp:1288: void CWallet::DecrementNoteWitnesses(const CBlockIndex*): Assertion `nWitnessCacheSize > 0' failed. when testing on regtest node.

[jan-matejka]

If I understand the issue correctly in my case it is caused by a fixture in my testsuite that "cleans" the chain between executing different testcases by invalidating block at height 1 which often causes invalidating a chain of more than 101 blocks. So increasing WITNESS_CACHE_SIZE by 1000 should solve the issue for me. Should I expect other issues with this patch?

[jan-matejka]

so I patched the MAX_REORG_LENGTH to increase it by 1000 but I still get the same error when trying to invalidateblock with more than 110 confirmations. Why is beyond me. Might be related to that I was unable to find where is the cache initialized by grepping for the WITNESS_CACHE_SIZE.

[nuttycom]

zcashd will no longer perform a reorg of more than 100 blocks