Hash Cache

[NicolasDorier]

Some notes about the implementation:

• It calculates the three midstate hashes as soon as a CheckSig segwit operation happens, whathever the SigHash,
• It is possible that two different threads calculate the midstate hashes of a transaction twice,

Befinits are:

• hashcashes map access is limited so we don't have too much lock contention
• Fewer conditional branches in consensus code
• Simple to review

This commit is only for having a cache that is simple to review and understand. It is probably possible to fix the two first points above, but the code overhead is not worth it when our goal is only to fix the O(n²) issue.

(rebased version of sipa#70)

[jl2012]

This one is hopefully merged for versions with segwit defined on mainnet

[dcousens]

You could avoid two look ups (IIRC) by comparing the size of the container before and after instead.

Aka:

auto sizeBefore = map.size();
map.insert(txId, hashes);
return map.size() != sizeBefore;

[NicolasDorier]

Actually, I don't think I really need TrySet to return a bool.

[dcousens]

hashcashes

hash caches?, had me confused for a second haha

utACK c2ea4dd

[dcousens]

trivial: maybe spacing between names?

[NicolasDorier]

@dcousens addressed your nits in dc188d8.

[dcousens]

utACK dc188d8

[jtimon]

Perhaps here we can do the same trick we did with the script/sigcache to keep script/interpreter simpler and more reusable.
ping @sipa

[jtimon]

Or maybe move SignatureHash to BaseSignatureChecker. Just brainstorming.

[sipa]

Assume a transaction has many signatures. One is SIGHASH_SINGLE, all the others are SIGHASH_ALL | SIGHASH_ANYONECANPAY.

The first computes and stores hashPrevouts. All the others will compute hashOutputs. However, after the first call, all TrySets will not do anything, as there is already a result in the cache, so it gets computed over and over again.

I think CachedHashes needs a Merge method like this:

void Merge(const CachedHashes& hashes) {
    if (hashPrevouts.IsNull()) hashPrevouts = hashes.hashPrevout;
    if (hashSequence.IsNull()) hashSequence = hashes.hashSequence;
    if (hashOutputs.IsNull()) hashOutputs = hashes.hashOutputs;
}

which can then be called from TrySet (instead of insert, use map[txid].Merge(hashes)).

[NicolasDorier]

so it gets computed over and over again

That's not true, I would be calculating the three hashes at the same time during the SIGHASH_SINGLE.
Take a look at the SignatureHash method, I changed it to calculate the three hashes aggressively.

I prefer not doing a merge. Without a merge my lock only have to protect the internal map as CachedHashes instances are read only. If we calculate the mid states lazily, I also need to be careful about locking at the CachedHashes instance level.

[sipa]

@NicolasDorier Oh, I see. I agree that the current code is fine in that case.

I don't understand the argument about the lock. The Merge function would also grab the lock, and be the only code that touches the map.

[NicolasDorier]

@sipa The Merge function as you did here can't grab the lock, because the lock is at the cache map level, not at the CachedHashes instance level.

But basically, if doing that way, I would need to grab the lock of the map around the Merge, as well as around any hash read of the HashedCaches instance. (so one can't read and call merge on the HashedCaches at the same time)
This would make lots of contention on a single lock. An alternative is to have one lock per HashedCaches... not sure if it is worth the complexity though, knowing that the only type of transaction which does not need the 3 midstate hashes are transaction which have no SIGHASH_ALL... this is very marginal case.

[NicolasDorier]

Closing this one in favor of #8422 which calculate hashes lazily.