Skip to content

Commit

Permalink
Faster bitmask iteration (#1228)
Browse files Browse the repository at this point in the history
* Add UnalignedBitChunks (#1227)

* Clippy

* Fix flaky test

* Improve test legibility

* Fix SlicesIterator offset direction

* Format

* Fix byte-aligned termination

* Test edge-cases

* More tests

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Review feedback

* Make UnalignedBitChunkIterator crate local

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
  • Loading branch information
tustvold and alamb committed Feb 2, 2022
1 parent 91d12ec commit f055fb0
Show file tree
Hide file tree
Showing 4 changed files with 547 additions and 166 deletions.
8 changes: 2 additions & 6 deletions arrow/src/buffer/immutable.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ use std::ptr::NonNull;
use std::sync::Arc;
use std::{convert::AsRef, usize};

use crate::util::bit_chunk_iterator::BitChunks;
use crate::util::bit_chunk_iterator::{BitChunks, UnalignedBitChunk};
use crate::{
bytes::{Bytes, Deallocation},
datatypes::ArrowNativeType,
Expand Down Expand Up @@ -205,11 +205,7 @@ impl Buffer {
/// Returns the number of 1-bits in this buffer, starting from `offset` with `length` bits
/// inspected. Note that both `offset` and `length` are measured in bits.
pub fn count_set_bits_offset(&self, offset: usize, len: usize) -> usize {
let chunks = self.bit_chunks(offset, len);
let mut count = chunks.iter().map(|c| c.count_ones() as usize).sum();
count += chunks.remainder_bits().count_ones() as usize;

count
UnalignedBitChunk::new(self.as_slice(), offset, len).count_ones()
}
}

Expand Down
Loading

0 comments on commit f055fb0

Please sign in to comment.