Skip to content

perf(store): remove dead sortedStore field from VersionIndexedStore#2820

Draft
pdrobnjak wants to merge 1 commit intoperf/skip-redundant-snapshotfrom
perf/remove-vis-sortedstore
Draft

perf(store): remove dead sortedStore field from VersionIndexedStore#2820
pdrobnjak wants to merge 1 commit intoperf/skip-redundant-snapshotfrom
perf/remove-vis-sortedstore

Conversation

@pdrobnjak
Copy link
Contributor

Summary

  • Remove the unused sortedStore *dbm.MemDB field from VersionIndexedStore
  • The field was allocated in the constructor but never read or written anywhere in the codebase
  • Eliminates ~4.2M dead MemDB + btree allocations per 30s benchmark window

Problem

VersionIndexedStore allocated a dbm.NewMemDB() in its constructor for a sortedStore field intended for future iterator support (per TODO comments). This field was never accessed — only declared and initialized. Each of the ~21 module stores per transaction created a VIS with this dead allocation.

Profiling after #2819 (30s, M4 Max, 1000 EVM transfers/block):

Metric Value
NewMemDB (via VIS) alloc_space (cum) 14.9 GB
btree.NewFreeListG alloc_space 15.3 GB (mostly from VIS)
VIS instances created per 30s ~4.2M (21 stores × 200K txs)

Changes

sei-cosmos/store/multiversion/mvkv.go — 3 lines removed:

  1. Field declaration: sortedStore *dbm.MemDB
  2. Constructor allocation: sortedStore: dbm.NewMemDB()
  3. Associated TODO comments

Benchmark Results (M4 Max, 1000 EVM transfers/block, 30s profile)

Metric Before (after #2819) After Delta
TPS (steady-state range) 7,200–8,600 8,400–9,400 median ~8,000 → ~9,000
NewMemDB (via VIS) alloc 14.9 GB cum 13.5 MB cum -14.9 GB (eliminated)
btree.NewFreeListG alloc 15.3 GB ~4.3 GB -11 GB
memclrNoHeapPointers CPU 10.71s 5.45s -49%
Idle CPU (usleep+kevent+pthread) 30.1s 41.6s +38% (workers finish faster)
Total CPU samples 132.26s 124.13s -6% (less total work needed)

pprof -alloc_space verification

NewMemDB (via VIS):  14,900 MB → 13.5 MB   (1100x reduction)
btree.NewFreeListG:  15,300 MB → 4,300 MB   (-11 GB)

CPU shift

 -5.26s  runtime.memclrNoHeapPointers  (fewer allocs to zero)
 +6.70s  runtime.usleep                (more idle — workers finish faster)
 +3.37s  runtime.kevent                (more idle)
 -1.32s  runtime.heapBitsSmallForAddr  (less heap metadata)

Test plan

  • go test ./sei-cosmos/store/multiversion/... -count=1 — pass
  • go test ./giga/tests/... -count=1 — pass (all 14 giga tests)
  • gofmt -s -w clean
  • Benchmark 2000+ blocks with heap + CPU profiling

🤖 Generated with Claude Code

@github-actions
Copy link

github-actions bot commented Feb 6, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedFeb 6, 2026, 6:42 PM

@codecov
Copy link

codecov bot commented Feb 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.67%. Comparing base (ae6bbe8) to head (f2970af).

Additional details and impacted files

Impacted file tree graph

@@                      Coverage Diff                      @@
##           perf/skip-redundant-snapshot    #2820   +/-   ##
=============================================================
  Coverage                         56.67%   56.67%           
=============================================================
  Files                              2031     2031           
  Lines                            165979   165978    -1     
=============================================================
+ Hits                              94061    94074   +13     
+ Misses                            63667    63660    -7     
+ Partials                           8251     8244    -7     
Flag Coverage Δ
sei-chain 41.54% <ø> (+0.01%) ⬆️
sei-cosmos 48.15% <ø> (-0.02%) ⬇️
sei-db 68.72% <ø> (ø)
sei-tendermint 58.08% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-cosmos/store/multiversion/mvkv.go 90.79% <ø> (-0.06%) ⬇️

... and 11 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pdrobnjak pdrobnjak force-pushed the perf/remove-vis-sortedstore branch from 0e0765c to 2f8f30d Compare February 6, 2026 12:04
@pdrobnjak pdrobnjak self-assigned this Feb 6, 2026
@pdrobnjak pdrobnjak force-pushed the perf/skip-redundant-snapshot branch from a3a9cf5 to 2286d83 Compare February 6, 2026 12:28
@pdrobnjak pdrobnjak force-pushed the perf/remove-vis-sortedstore branch from 2f8f30d to 024e61c Compare February 6, 2026 12:28
@pdrobnjak pdrobnjak marked this pull request as draft February 6, 2026 15:25
@pdrobnjak pdrobnjak force-pushed the perf/skip-redundant-snapshot branch from 2286d83 to 64fcc93 Compare February 6, 2026 17:20
@pdrobnjak pdrobnjak force-pushed the perf/remove-vis-sortedstore branch from 024e61c to 19fc5b8 Compare February 6, 2026 17:20
@pdrobnjak pdrobnjak force-pushed the perf/skip-redundant-snapshot branch from 64fcc93 to e91f6e4 Compare February 6, 2026 17:43
@pdrobnjak pdrobnjak force-pushed the perf/remove-vis-sortedstore branch from 19fc5b8 to f6066c0 Compare February 6, 2026 17:43
@pdrobnjak pdrobnjak force-pushed the perf/skip-redundant-snapshot branch from e91f6e4 to ae6bbe8 Compare February 6, 2026 18:41
The sortedStore *dbm.MemDB field in VersionIndexedStore was allocated in
the constructor (NewVersionIndexedStore) but never read or written
anywhere in the codebase. Each VIS instance allocated a full MemDB with
btree + FreeList — ~21 VIS per transaction × ~200K txs in 30s = ~4.2M
dead MemDB allocations.

Remove the field and its allocation entirely.

Results (M4 Max benchmark, diff vs previous):
- NewMemDB via VIS: 14.9 GB → 13.5 MB (eliminated)
- btree.NewFreeListG: 15.3 GB → ~4.3 GB (-11 GB)
- memclrNoHeapPointers CPU: 10.71s → 5.45s (-49%)
- TPS: 7,200-8,600 → 8,400-9,400 (median ~8,000 → ~9,000)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@pdrobnjak pdrobnjak force-pushed the perf/remove-vis-sortedstore branch from f6066c0 to f2970af Compare February 6, 2026 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant