Skip to content

Conversation

@kcalvinalvin
Copy link
Member

1: Switch the dbcache to use mutable treaps.

ffldb was using a immutable treap for the database cache. While this is
good in that it supports snapshotting of the database transactions,
there is no need for this as a generic Bitcoin block storage.

Even worse, the immutable treap would allocate O(n) nodes per insert and
deletion, leading to a huge amount of memory being allcated.

This PR switches away from immutable treaps as there isn't really a need
for snapshotting and it costs too much in terms of memory allocation.

2: Save spend journals as flat files.

Spend journals are needed during reorgs, which rarely happens. These
spend journals were being stored in leveldb, which isn't necessary.
This PR makes the spend journals to be saved in flat files, just like
blocks.

There now isn't any support for deleting spend journals but this isn't
really a concern. The current btcd model of storing blocks is if the
received block passes initial pow and sanity checks. Since spend
journals are only created when the block passes all verification, it's
even less likely that the spend journals are effected by disk fill
attacks.

There isn't any need for deleting blocks and so there is even less of a
need for deleting spend journals.

@coveralls
Copy link

coveralls commented Nov 8, 2021

Pull Request Test Coverage Report for Build 1433731997

  • 189 of 245 (77.14%) changed or added relevant lines in 5 files are covered.
  • 11 unchanged lines in 5 files lost coverage.
  • Overall coverage increased (+0.09%) to 54.147%

Changes Missing Coverage Covered Lines Changed/Added Lines %
database/ffldb/blockio.go 42 43 97.67%
blockchain/chainio.go 5 9 55.56%
database/ffldb/db.go 103 122 84.43%
database/ffldb/reconcile.go 24 56 42.86%
Files with Coverage Reduction New Missed Lines %
database/ffldb/db.go 1 91.66%
database/ffldb/reconcile.go 1 53.21%
database/ffldb/dbcache.go 2 76.53%
integration/rpctest/blockgen.go 2 83.67%
peer/peer.go 5 75.06%
Totals Coverage Status
Change from base Build 1433728782: 0.09%
Covered Lines: 25580
Relevant Lines: 47242

💛 - Coveralls

1: Switch the dbcache to use mutable treaps.

ffldb was using a immutable treap for the database cache.  While this is
good in that it supports snapshotting of the database transactions,
there is no need for this as a generic Bitcoin block storage.

Even worse, the immutable treap would allocate O(n) nodes per insert and
deletion, leading to a huge amount of memory being allcated.

This PR switches away from immutable treaps as there isn't really a need
for snapshotting and it costs too much in terms of memory allocation.

2: Save spend journals as flat files.

Spend journals are needed during reorgs, which rarely happens.  These
spend journals were being stored in leveldb, which isn't necessary.
This PR makes the spend journals to be saved in flat files, just like
blocks.

There now isn't any support for deleting spend journals but this isn't
really a concern.  The current btcd model of storing blocks is if the
received block passes initial pow and sanity checks.  Since spend
journals are only created when the block passes all verification, it's
even less likely that the spend journals are effected by disk fill
attacks.

There isn't any need for deleting blocks and so there is even less of a
need for deleting spend journals.
@kcalvinalvin kcalvinalvin merged commit d2ea7fc into utreexo Nov 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants