Skip to content

feat: add benchmarks, expanded proptest, and cargo-fuzz targets#9

Merged
prestwich merged 1 commit intomainfrom
prestwich/benchmarks-and-fuzzing
Apr 7, 2026
Merged

feat: add benchmarks, expanded proptest, and cargo-fuzz targets#9
prestwich merged 1 commit intomainfrom
prestwich/benchmarks-and-fuzzing

Conversation

@prestwich
Copy link
Copy Markdown
Member

@prestwich prestwich commented Mar 28, 2026

Summary

  • Benchmarks: Add 5 new benchmark files — cursor writes, reserve vs put, nested transactions, concurrency (up to 128 readers), and scaling (100 to 100K entries × 32/128/512B values). 32-byte keys, 128-byte values, parameterized value sizes, standardized entry counts.
  • Proptest: Split monolithic proptest_inputs.rs into 6 domain-focused files (kv, cursor, dupsort, dupfixed, iter, nested). Migrate all existing tests and add new cases for large values, multi-db isolation, DUPFIXED page spanning, nested txn semantics, and more. DUPFIXED tests cover both sync and unsync transactions.
  • Fuzz: Set up cargo-fuzz with 6 targets focused on FFI/unsafe boundaries — Cow<[u8]> decode paths, fixed-size array decode, dirty page roundtrip, DUPFIXED page decode (with iterator exercise), and key validation. Debug assertions enabled in fuzz release profile.
  • Input validation docs: Document the debug-only validation model prominently in README, lib.rs rustdoc, and CLAUDE.md. MDBX aborts the process on certain constraint violations; our debug assertions catch these during development, while release builds trust the caller.
  • Cleanup: Remove dead return-borrowed feature documentation from iter module. Remove all evmdb/parity references.

Test plan

  • All unit tests pass (cargo t)
  • All proptest files pass (68+ property tests, including sync tx variants)
  • All benchmarks run clean (cargo bench)
  • Clippy clean (--all-targets)
  • cargo +nightly fmt applied
  • Docker Linux tests pass
  • Fuzz targets compile and run clean for 10s each (cargo +nightly fuzz run <target>)
  • Doc tests pass

🤖 Generated with Claude Code

@prestwich prestwich force-pushed the prestwich/benchmarks-and-fuzzing branch from e800bef to 2ac4f42 Compare March 31, 2026 14:51
@prestwich
Copy link
Copy Markdown
Member Author

[Claude Code]

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@prestwich prestwich requested review from Evalir and Fraser999 April 6, 2026 17:11
Copy link
Copy Markdown
Member

@Evalir Evalir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a lot of tests :D the docs make sense, and i think the debug-based validation approach makes sense

Copy link
Copy Markdown
Contributor

@Fraser999 Fraser999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude made the following assessment of the parity benchmarks:

Critical mismatches (results not comparable)

cursor_write::put::sync vs write_put_100

Aspect mdbx evmdb
Txn creation In setup (untimed) In timed path
Commit Never (aborts) Commits every iteration
Key ordering Sorted, same every iteration Random XOR-scrambled, different every iteration
DB state across iterations Empty (abort resets) Grows (commits accumulate)

cursor_write::append::sync vs write_put_100_sorted

All the same mismatches as put::sync vs write_put_100, plus:

Aspect mdbx evmdb
Operation cursor.append() (MDBX_APPEND flag) Regular put() in sorted order

These are fundamentally different operations - append is an optimized path that skips B+tree traversal.

cold_sequential_scan (mdbx) vs cold_sequential_scan (evmdb)

Aspect mdbx evmdb
Operation Cursor scan (cursor.next() loop) N individual point lookups (get_blocking)
Entry count 1,000,000 10,000
Env lifecycle Reopens environment each iteration Keeps DB open, flushes cache

evmdb's version is not actually a "scan" - it's N independent B+tree traversals from root.

cold_random_get (mdbx) vs cold_random_get (evmdb)

Aspect mdbx evmdb
Entry count 1,000,000 10,000
Lookup count 1,000 100
Env lifecycle Reopens environment each iteration Keeps DB open, flushes cache

Significant mismatches

append_ordered_put vs put_sorted

Aspect mdbx evmdb
Operation txn.append() (MDBX_APPEND) Regular put() in sorted order
Commit batching Single commit for all N entries Batches of 1000, commit after each batch

readers_no_writer and readers_one_writer

Aspect mdbx evmdb
Reader work Reads value, accumulates val.len() Checks is_some() only

sequential_get and random_get

Aspect mdbx evmdb
Reader work Reads value via Cow<[u8]>, accumulates val.len() Checks is_some() only

Minor

  • Entry counts differ by default for all scaling benchmarks: mdbx omits 100K without BENCH_FULL=1; evmdb always includes 100K.

Comment thread CLAUDE.md Outdated
Comment thread benches/utils.rs Outdated
Comment thread fuzz/Cargo.toml
Comment thread fuzz/fuzz_targets/dupfixed_page_decode.rs
Comment thread tests/proptest_dupfixed.rs
- Benchmarks: Add cursor writes, reserve vs put, nested transactions,
  concurrency (up to 128 readers), and scaling (100 to 100K entries
  × 32/128/512B values).
- Proptest: Split monolithic proptest_inputs.rs into 6 domain-focused
  files (kv, cursor, dupsort, dupfixed, iter, nested). DUPFIXED tests
  cover both sync and unsync transactions.
- Fuzz: Set up cargo-fuzz with 6 targets focused on FFI/unsafe
  boundaries. Debug assertions enabled in fuzz release profile.
- Document debug-only input validation model in README and lib.rs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@prestwich prestwich force-pushed the prestwich/benchmarks-and-fuzzing branch from c3148b9 to 02c20b4 Compare April 7, 2026 14:24
@prestwich prestwich merged commit 90cc1e2 into main Apr 7, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants