[storage] Balance Storage Overhaul

[patrick-ogrady]

Closes: #247

This PR introduces significant optimizations to BalanceStorage that dramatically speed up sync speed in the rosetta-cli (mostly through the reduction of lock contention). Special shoutout to @yflinn, @sidhujag, and @HelloKashif for debugging the issues that led to this PR.

Changes

• Use big.Int bytes format to store amount
• Don't use txn.Scan when updating balance
• cache most recent balance
  • update during orphan
• Don't store pruning index, last reconciled index, and current balance on same record
• Count accounts seen and accounts reconciled
  • Use in EstimatedReconciliationCoverage
  • Update accounts reconciled in block syncing loop so that account txs don't conflict
• Rebase on [storage] Interface Refactor + Test Cleanup #254
  • squash commits
• Add tests (blocked by [storage] Interface Refactor + Test Cleanup #254)
  • Pass existing tests
  • Add test for estimated reconciliation coverage
• Rollback counts when deleting account records (conditionally remove reconciliation if exists)
• Don't error if reconciled orphaned account (could be a race where we delete account while reconciling)

Future Work (before release)

• limit concurrent goroutines in syncing to NumCPU (pretty good default) instead of unbounded (can cause CPU to freeze or OOM)
  • Use: https://github.com/neilotoole/errgroup
• pre-allocate allChanges in parser (SDK updates for performance, rework BalanceStorage::UpdateBalance #247 )

[coveralls]

Pull Request Test Coverage Report for Build 12642

• 211 of 278 (75.9%) changed or added relevant lines in 2 files are covered.
• 8 unchanged lines in 1 file lost coverage.
• Overall coverage decreased (-0.08%) to 78.124%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
storage/modules/counter_storage.go	14	15	93.33%
storage/modules/balance_storage.go	197	263	74.9%

Files with Coverage Reduction	New Missed Lines	%
storage/modules/balance_storage.go	8	79.08%

Totals
Change from base Build 12487:	-0.08%
Covered Lines:	7853
Relevant Lines:	10052

---

💛 - Coveralls

[HelloKashif]

Hi @patrick-ogrady
Do you have timeline on this PR completion?

[patrick-ogrady]

Hi @patrick-ogrady
Do you have timeline on this PR completion?

Looking to have this all merged and released by Tuesday (ended up having to do a small refactor to clean up testing -> #254 )! Thanks for your patience!