Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduced overhead of MDB_RESERVE #1

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

nolanderc
Copy link

See also my changes here https://github.com/nolanderc/heed/tree/reserved-space.

  • use #[inline] on ReservedSpace to enable inlining across crates. It is also possible to use lto = "thin" or lto = true under [profile.release].
  • Made it possible to write into &[MaybeUninit<u8>] without UB using UninitWriter.

Seems std::io::Write for &mut [u8] is slower than expected. In my experiments, moving the check for buf.len() > space.remaining() as early as possible in the write function improved performance drastically (2x faster).

Test results on my laptop:

$ ./run.sh
+ ./target/release/heed-perfs-put-reserved classic-codec
  19.61s [insert:    8.04s, commit:   11.57s]
+ ./target/release/heed-perfs-put-reserved put-reserved
  19.37s [insert:    7.74s, commit:   11.62s]
+ ./target/release/heed-perfs-put-reserved put-reserved-alloc
  20.05s [insert:    8.83s, commit:   11.22s]
+ ./target/release/heed-perfs-put-reserved put-reserved-uninit
  18.57s [insert:    6.66s, commit:   11.91s]
+ ./target/release/heed-perfs-put-reserved put-reserved-uninit-fill-zeroes
  19.00s [insert:    7.10s, commit:   11.90s]
+ ./target/release/heed-perfs-put-reserved put-reserved-uninit-into-slice
  26.32s [insert:   15.40s, commit:   10.92s]

There's not a huge improvement over just pre-allocating a Vec, but yielded a 1.6s reduction in insertion time at its best. But this is a small benchmark where all data is the same size, which is also done on my laptop, so YMMV.

- use `#[inline]` on `ReservedSpace`
- write into `&[MaybeUninit<u8>]` without UB

Seems `std::io::Write for &mut [u8]` is slower than expected.
In my experiments, moving the check for `buf.len() > space.remaining()`
as early as possible in the `write` function improved performance
dramatically.

Test results on my laptop:

    + ./target/release/heed-perfs-put-reserved classic-codec
      19.61s [insert:    8.04s, commit:   11.57s]
    + ./target/release/heed-perfs-put-reserved put-reserved
      19.37s [insert:    7.74s, commit:   11.62s]
    + ./target/release/heed-perfs-put-reserved put-reserved-alloc
      20.05s [insert:    8.83s, commit:   11.22s]
    + ./target/release/heed-perfs-put-reserved put-reserved-uninit
      18.57s [insert:    6.66s, commit:   11.91s]
    + ./target/release/heed-perfs-put-reserved put-reserved-uninit-fill-zeroes
      19.00s [insert:    7.10s, commit:   11.90s]
    + ./target/release/heed-perfs-put-reserved put-reserved-uninit-into-slice
      26.32s [insert:   15.40s, commit:   10.92s]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant