WIP: MMAP backed bucket map that is suitable as a KV store for the accounts index #18179

aeyakovenko · 2021-06-23T16:15:09Z

Problem

Accounts index takes up way to much RAM

Summary of Changes

Design

create a large set of buckets, 32k or more up front
pubkeys based on their first 16 bits are stored in each bucket
each bucket has a mmap for the index entry and N mmaps for variable sized [SlotInfo] data
each data mmap stores a 2^n SlotInfo slices

Each bucket index or bucket data can double independently.

Locking

Each bucket has its own RWLock. We should test performance with spin::RwLock vs sync::RwLock. The DataBucket allocator can function with atomics, so it's possible to redesign this such that the write lock is only necessary when reallocating.

In Order Scan

The buckets are in order, while keys in each bucket are going to be out of order. To scan the entire index in order, read each bucket in order and use a rayon parallel sort.

Fixes #

tag @carllin @jeffwashington @sakridge @ryoqun

jeffwashington · 2021-06-23T16:46:04Z

I have done several passes of test code that could be helpful here to gather metrics across # accounts, insert, lookup, memory usage for hashmap, dashmap, btree, binned btree, skip-list and binned skip-list. So far, I've been focusing on the various in-memory representations.

jon-chuang

Just doing some basic sanity checking

jon-chuang · 2021-06-24T20:42:36Z

runtime/src/data_bucket.rs

+    path: PathBuf,
+    mmap: MmapMut,
+    pub cell_size: u64,
+    pub capacity: u64,


Could you clarify if capacity is in units of cell_size, or bytes?
Edit: I believe it’s cell_size.

Yea. num_cells would be better :)

jon-chuang · 2021-06-24T20:42:36Z

runtime/src/data_bucket.rs

+        let drive = &drives[r];
+        let pos = format!("{}", thread_rng().gen_range(0, u128::MAX),);
+        let file = drive.join(pos);
+        let mut data = OpenOptions::new()


I guess in the future this could recycle old MMaps instead of remove_file below

jon-chuang · 2021-06-24T20:42:36Z

runtime/src/bucket_map.rs

+            for i in self.data.len() as u64..(sz.0 + 1) {
+                self.data.push(DataBucket::new(
+                    self.drives.clone(),
+                    (1 << i) + MIN_ELEMS,


Why do you grow the DataBucket cell_size exponentially?

I guess it is to fit the data.len() as per best_fit_bucket

aeyakovenko · 2021-06-26T01:35:24Z

Some benchmarks. So single insert isn't super optimized yet, but its still pretty good.

on my intel mac

test bucket_map_bench_hashmap_baseline       ... bench:         269 ns/iter (+/- 134)
test bucket_map_bench_insert_1               ... bench:         770 ns/iter (+/- 116)
test bucket_map_bench_insert_16x512          ... bench:  15,533,122 ns/iter (+/- 44,959,341)
test bucket_map_bench_insert_16x512_baseline ... bench:  23,809,525 ns/iter (+/- 4,480,271)

it seems to start outperform the RwLock<HashMap> with 16+ threads/buckets and 256+ items per thread

aeyakovenko · 2021-06-26T05:12:57Z

4096 buckets on Lemond with each thread adding 4096 items:

running 1 test
time: 471ns per item
dir_size: "/tmp/bucket_map_test_mt" 2712141824
dir_size: "/mnt/data/aeyakovenko/bucket_map_test_mt" 2774007808

Overhead per account: (2774007808 + 2712141824)/(4096 * 4096) = 327 bytes
(that's pretty decent I think)

aeyakovenko · 2021-06-27T14:01:47Z

runtime/src/accounts_db.rs

-            keys.par_chunks(chunks)
+            let mut hashes = vec![];
+            for ix in 0..self.accounts_index.account_maps.num_buckets() {
+                let keys = self.accounts_index.account_maps.keys(ix);


@ryoqun @sakridge @jeffwashington

wdyt, would this work here? the initial design had a snapshot slot that would keep the data behind that slot form being cleaned up as the index is updated. So we shouldn't need a global lock on all the keys to do the snapshot.

@brooksprumo is good to reference here, as he's been working on snapshotting related logic

I think this code path is already dead except for debug validation. We don’t want to use account index for hash calculation.
Even so, does bucket[x] have keys that are all < the keys in bucket[x+1]? This particular algorithm is written to assume the keys vector is in sorted order. The btree provides that currently. If not, we have to save key and hash from this scan, sort by key, then hash.

I can confirm what @jeffwashington said that on the snapshot side this is never called/used. The snapshot code that calls AccountsDb::calculate_accounts_hash_without_index() is inside an if that is always false.

There are hooks for testing, but none of the tests actually exercise this code path. I assume in the past this was different, but as of now AccountsDb::calculate_accounts_hash_without_index() is never called in the snapshot code (nor anything that calls these snapshot functions).

Relevant snapshot functions:
snapshot_utils::snapshot_bank()
snapshot_utils::package_snapshot()

@jeffwashington

Even so, does bucket[x] have keys that are all < the keys in bucket[x+1]?

Yep. that is guaranteed. https://github.com/solana-labs/solana/pull/18179/files#diff-d90ce8cbcea9459dbca60678b48f6d6d2816b8d4de8b8ebfd5c434b819f153b3R97

At boot, the validator creates a fixed number of buckets. On lemond with 128 cores, I see performance improve up to 4096 buckets.

jeffwashington · 2021-06-30T20:50:28Z

fyi, I am working on this

jeffwashington · 2021-06-30T20:51:02Z

@aeyakovenko , @sakridge
fyi that pubkeys are almost perfectly uniform currently on mnb - at least into the highest order 16 bins.

jeffwashington · 2021-06-30T21:15:13Z

ledger-tool verify test

wiggins, mnb, 61M accounts, 71M accounts with duplicates, single normal folder for files
master:
generate_index total_us= 16595216i scan_stores_us= 16823797i
generate_index total_us=825187084i scan_stores_us=276917451i
new accounts_index^

stale · 2021-07-08T01:27:47Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

stale · 2021-07-15T11:12:24Z

This stale pull request has been automatically closed. Thank you for your contributions.

jeffwashington · 2021-07-21T17:38:01Z

just a comment here since its stale. This is still actively in prototyping.

aeyakovenko added 11 commits Jun 3, 2021

updates

e8eb031

gnarly rwlocks

42bfa0e

progress

541d25b

update

c766aa0

update

e49ada5

wip

7f565cb

wip

77c3834

wip

ad9dea0

update

ce552ac

wip

7cb5651

update

9810bd2

aeyakovenko changed the title ~~SSD backed bucket map that is suitable as a KV store for the accounts index~~ WIP: MMAP backed bucket map that is suitable as a KV store for the accounts index Jun 23, 2021

aeyakovenko added 3 commits Jun 23, 2021

cleanup

1e79b49

remove update error

1e71b88

cleanup

1b0ce49

aeyakovenko added 6 commits Jun 23, 2021

update

d36c50b

new func for bucketmap

3df3835

bugs

4791c18

update

18b7098

update

5f1635b

update

44c0051

jon-chuang reviewed Jun 24, 2021

View changes

aeyakovenko added 4 commits Jun 25, 2021

wip

0eae241

wip

6b22270

woot

3d0dc73

update

c67c0f4

aeyakovenko added 2 commits Jun 26, 2021

benches

1bc7358

benches

766f054

move to sep package

37b8896

aeyakovenko force-pushed the bucket_map branch from c98d3c5 to 37b8896 Compare Jun 26, 2021

aeyakovenko added 4 commits Jun 26, 2021

missing

d48d7ff

cleanup

690ad42

updates

67c990d

fixed

0634622

aeyakovenko added 4 commits Jun 26, 2021

update

7f3ef49

cleanup

55265da

cleanup

80ae95f

unlocked collected hashes

66b967b

aeyakovenko commented Jun 27, 2021

View changes

aeyakovenko and others added 4 commits Jun 27, 2021

update

ae5d915

update

75c9316

addref calls

de9bcf8

fmt

ad91e2e

stale bot added the stale label Jul 8, 2021

stale bot closed this Jul 15, 2021

carllin mentioned this pull request Jul 23, 2021

Design for on-disk accounts index #18854

Open

WIP: MMAP backed bucket map that is suitable as a KV store for the accounts index #18179

WIP: MMAP backed bucket map that is suitable as a KV store for the accounts index #18179

aeyakovenko commented Jun 23, 2021 •

edited

jeffwashington commented Jun 23, 2021

jon-chuang left a comment

jon-chuang Jun 24, 2021

aeyakovenko Jun 26, 2021

jon-chuang Jun 24, 2021

aeyakovenko Jun 26, 2021

jon-chuang Jun 24, 2021

jon-chuang Jun 24, 2021

aeyakovenko commented Jun 26, 2021 •

edited

aeyakovenko commented Jun 26, 2021

aeyakovenko Jun 27, 2021

carllin Jun 27, 2021

jeffwashington Jun 28, 2021 •

edited

brooksprumo Jun 28, 2021

brooksprumo Jun 28, 2021

aeyakovenko Jun 28, 2021 •

edited

jeffwashington commented Jun 30, 2021

jeffwashington commented Jun 30, 2021 •

edited

jeffwashington commented Jun 30, 2021 •

edited

stale bot commented Jul 8, 2021

stale bot commented Jul 15, 2021

jeffwashington commented Jul 21, 2021

WIP: MMAP backed bucket map that is suitable as a KV store for the accounts index #18179

WIP: MMAP backed bucket map that is suitable as a KV store for the accounts index #18179

Conversation

aeyakovenko commented Jun 23, 2021 • edited

Problem

Summary of Changes

Design

Locking

In Order Scan

jeffwashington commented Jun 23, 2021

jon-chuang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aeyakovenko commented Jun 26, 2021 • edited

aeyakovenko commented Jun 26, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jeffwashington Jun 28, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aeyakovenko Jun 28, 2021 • edited

Choose a reason for hiding this comment

jeffwashington commented Jun 30, 2021

jeffwashington commented Jun 30, 2021 • edited

jeffwashington commented Jun 30, 2021 • edited

stale bot commented Jul 8, 2021

stale bot commented Jul 15, 2021

jeffwashington commented Jul 21, 2021

aeyakovenko commented Jun 23, 2021 •

edited

aeyakovenko commented Jun 26, 2021 •

edited

jeffwashington Jun 28, 2021 •

edited

aeyakovenko Jun 28, 2021 •

edited

jeffwashington commented Jun 30, 2021 •

edited

jeffwashington commented Jun 30, 2021 •

edited