Add FlatKV database to the benchmarking utility.#2936
Conversation
|
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
|
|
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
|
|
Do you mind also updating the readme? Thanks! |
| cfg := flatkv.DefaultConfig() | ||
| cs := flatkv.NewCommitStore(dir, logger.NewNopLogger(), cfg) | ||
|
|
||
| _, err := cs.LoadVersion(0, false) |
There was a problem hiding this comment.
Need to remove the extra arg
| // varying keysPerBlock and numBlocks to find optimal block size. | ||
| func BenchmarkWriteWithDifferentBlockSize(b *testing.B) { | ||
| func BenchmarkMemIAVLWriteWithDifferentBlockSize(b *testing.B) { | ||
| const totalKeys int64 = 1_000_000 |
There was a problem hiding this comment.
Let's change this to 100k as well?
There was a problem hiding this comment.
Oh nvm, it seems 100k isn't accurate enough, we probably wanna test more
There was a problem hiding this comment.
For the total testing time perspective, maybe still reduce to 100k level, and use a smaller batch size like 1,2,10,20 and up till 100,200?
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2936 +/- ##
=======================================
Coverage 57.76% 57.77%
=======================================
Files 2111 2111
Lines 174235 174217 -18
=======================================
Hits 100649 100649
+ Misses 64631 64613 -18
Partials 8955 8955
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
|
seems like the current benchmarks have high variance, I get different numbers each time. Is there a way to reduce or eliminate this variability? |
Readme updated |
My first instinct on this is that the regular suite of benchmarks is very short for a DB test. In the past when I've done DB benchmarking, I often observe that it takes minutes or hours for DB performance to reach steady state performance. Some of these tests are completing in ~15 seconds on my laptop. Good for a first pass approximation, but insufficient for a high fidelity reading. |
We can have a separate PR to address that issue, I don't think it's a blocker for this PR |
Ticket: https://linear.app/seilabs/issue/STO-358/state-db-go-bench-should-testing-all-sc-backends Add support to the DB benchmarking utility for different DB implementations. I ran the benchmarks locally and verified that they worked as expected. --------- Co-authored-by: Cody Littley <cody.littley@seinetwork.io>
3 participants
Describe your changes and provide context
Ticket: https://linear.app/seilabs/issue/STO-358/state-db-go-bench-should-testing-all-sc-backends
Add support to the DB benchmarking utility for different DB implementations.
Testing performed to validate your change
I ran the benchmarks locally and verified that they worked as expected.