Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have BitField store ranges instead of bytes, and add benchmarks #543

Merged
merged 21 commits into from
Jul 17, 2020

Conversation

timvermeulen
Copy link
Contributor

The main change: remove the RlePlus struct that held RLE+ encoded data, and instead have BitField store Vec<Range<usize>> directly. Specifically:

  • when deserializing a bit field, instead of only verifying the data and keeping it as RLE+ encoded bytes, we verify the data by decoding it to Vec<Range<usize>> and store those ranges instead
  • when iterating a bit field, the data no longer needs to be decoded on the fly — the ranges are iterated directly

As a result, deserializing a bit field is now about 1/3 slower, but deserializing + iterating a bit field is about 2x faster (and every subsequent iteration is much faster as well).


Furthermore, improved ergonomics: added a Peekable struct just like std::iter::Peekable, and have the Union / Intersection / Difference iterators store those instead of storing the iterators and next ranges separately. Peekable distinguishes between not having looked at the next range and there not being a next range, so this allowed us to remove all FusedIterator bounds.

Other changes:

  • add a couple of benchmarks that we can measure future changes with
  • BitField::first now has a custom implementation that does not allocate
  • BitField::get is made a lot faster by doing a binary search on the range vector

iter: I,
/// Stores the peeked range. `None` means that no range was peeked, and
/// `Some(None)` means that `peek` was called but the iterator was empty.
peeked: Option<Option<Range<usize>>>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optionception 🧠 💥

@timvermeulen timvermeulen merged commit 916bd4a into main Jul 17, 2020
@timvermeulen timvermeulen deleted the bitfield-benchmarks branch July 17, 2020 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants