Skip to content

Compute 64-bit Xor in place - #565

Open
gitRasheed wants to merge 3 commits into
RoaringBitmap:masterfrom
gitRasheed:perf/xor64-inplace
Open

Compute 64-bit Xor in place#565
gitRasheed wants to merge 3 commits into
RoaringBitmap:masterfrom
gitRasheed:perf/xor64-inplace

Conversation

@gitRasheed

Copy link
Copy Markdown

Description

Includes the two commits of #564, which this change needs: the in-place path calls the 32-bit Xor, and before that fix it modifies its argument. The diff reduces to one commit once #564 merges.

roaring64.Bitmap.Xor allocated a fresh symmetric difference for every matching key and swapped it into the receiver. It now XORs into the receiver's writable container, as the 32-bit Xor does. Unshared dense containers stop allocating.

Type of Change

  • Performance improvement

Changes Made

What was changed?

  • roaring64/roaring64.go: getWritableContainerAtIndex plus the 32-bit in-place Xor, replacing roaring.Xor.
  • roaring64/xor_inplace_test.go: zero allocations on dense reuse, and benchmarks.

Why was it changed?

It was the only in-place set operation here that still allocated the result for every matching key.

How was it changed?

Two lines.

Testing

go test ./... passes on arm64 and amd64. The ownership test from #564 covers every container pair.

Formatting

go fmt clean.

Performance Impact

Against #564 on c8g.xlarge, medians of ten one-second samples on one pinned core, benchstat p < 0.001 unless marked. Each synthetic input has four keys. A reuse op is two XORs that restore the receiver; a clone-and-xor op is one clone and one XOR. Each real-data op clones the left bitmap and XORs it with the right for all 199 adjacent pairs of the dataset, lifted to one 64-bit prefix so the keys match.

Case before ns/op after ns/op time allocs/op
dense/reuse 16,631 5,179 -69% 48 -> 0
dense/clone-and-xor 13,686 8,238 -40% 57 -> 33
array/reuse 2,467 1,700 -31% 48 -> 16
array/clone-and-xor 2,245 1,847 -18% 57 -> 41
run/reuse 31,180 19,650 -37% 64 -> 16
run/clone-and-xor 23,890 17,740 -26% 73 -> 49
disjoint-keys/reuse 1,310 1,266 -3% 40 -> 36
disjoint-keys/clone-and-xor 1,705 1,706 0% (p=0.84) 64 -> 64
census-income 3,067,758 2,877,270 -6% 6,110 -> 4,801
census1881 1,714,383 1,230,372 -28% 12,003 -> 8,187
weather_sept_85 11,220,297 10,330,852 -8% 15,942 -> 13,103

After RunOptimize the three datasets improve by 6%, 26% and 8%.

Breaking Changes

None.

An array or run receiver XORed with a bitmap container ran the in-place
operation on the argument's container and adopted it, so a.Xor(b)
changed b and left both bitmaps sharing storage. Compute the result on
the receiver's side instead. The test covers every in-place operation
and container pair.
Keys present only in the argument were inserted by pointer, so editing
the receiver afterwards changed the argument. Clone them, as Or does.
XORing a bitmap with itself removed entries from the structure being
iterated and could panic; clear the receiver instead, as the 32-bit Xor
does.
Reuse the receiver's writable container instead of allocating a fresh
symmetric difference for every matching key.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant