ledger: move spver context hash calculation out from catchpoint writing

[algorandskiy]

Summary

1. stateproof verification snapshot logic does not handle CatchpointTracking=1 properly - its hash calculation is a side effect of writing spver data chunk into catchpoint file. Fixed that by calculating hash similarly to balance hash on demand.
2. Improve catchpointdump to allow rebuilding MT and print accounts and stateproof hashes.

Test Plan

1. Modified unit test for CatchpointTracking=1
2. Tested locally

[algonautshant]

LGTM

[codecov]

Codecov Report

Merging #5574 (85f8690) into master (d316914) will increase coverage by 0.02%.
The diff coverage is 65.11%.

@@            Coverage Diff             @@
##           master    #5574      +/-   ##
==========================================
+ Coverage   55.79%   55.81%   +0.02%     
==========================================
  Files         446      446              
  Lines       63418    63425       +7     
==========================================
+ Hits        35381    35401      +20     
+ Misses      25668    25656      -12     
+ Partials     2369     2368       -1     

Impacted Files	Coverage Δ
ledger/catchpointwriter.go	62.34% <50.00%> (ø)
ledger/catchupaccessor.go	64.80% <62.50%> (+<0.01%)	⬆️
ledger/catchpointtracker.go	59.64% <76.92%> (-0.05%)	⬇️

... and 7 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[algonautshant]

LGTM

[jsgranados]

Just a suggestion, but the ledger readme doesn't mention the stateproof tracker. Should we add that?

[cce]

You are adding an extra DB call to GetAllSPContexts here, to get the hash and drop the data. Then later in WriteStateProofVerificationContext() the DB is called again to get the data but throw away the hash. I guess it is safe to assume the DB will return the same data/hash between the two calls (is it possible it won't?). The data is written out when finishFirstStage is called by finishFirstStageAfterCrash for example and I wasn't sure whether there are cases where things can get mixed up between the two DB calls to get the SP data..

I'm looking at the code now trying to figure out, is there a way to have the hash and data be generated from the same DB call.. just to have one less thing to worry about getting out of sync

[cce]

Or maybe if you just have if !ct.enableGeneratingCatchpointFiles around this?

[algorandskiy]

Well there is the same "issue" with accounts - they are written into a file in generateCatchpointData and hash is calculated later in recordFirstStageInfo by reading MT from DB. There is no difference in approach.

[cce]

What about a smaller change like this 7b28351

[algorandskiy]

could work, why not

[algonautshant]

Both calls come from lt.postCommitUnlocked, and the database is changed in lt.commitRound sequentially before the postCommitUnlocked call.

[cce]

What do you think of #5579?

[algorandskiy]

replaced by #5579