Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3 points on 2 segments Xor8Plus variant #11

Closed
wants to merge 3 commits into from

Conversation

@funny-falcon
Copy link
Contributor

funny-falcon commented Dec 26, 2019

This PR is to demonstrate alternative points placement to save memory lookup in Xor8Plus variant.

It trades a bit of BitsPerValue (9.17 vs 9.13 for 3 segment) of a Xor8Plus
variant for collocating first two lookups in a single cache line.

(this PR contains 1 commit over #10)

It trades a bit of BitsPerValue (9.17 vs 9.13 for 3 segment) of a Xor8Plus
variant for collocating first two lookups in a single cache line.

# Conflicts:
#	xorfilter.go
@funny-falcon funny-falcon force-pushed the funny-falcon:3points2segments branch from 0c624e7 to c7b541f Dec 26, 2019
@funny-falcon

This comment has been minimized.

Copy link
Contributor Author

funny-falcon commented Dec 26, 2019

Actually 9.13 vs 9.17 is for non-seeded random generator.
If random generator is seeded with current time (rand.Seed(time.Now().UnixNano()), it have same average BitsPerValue for "plus" variant as the original one.

@funny-falcon

This comment has been minimized.

Copy link
Contributor Author

funny-falcon commented Dec 26, 2019

If cache locality doesn't matter, ie if first two points could be anywhere in first segment, then Bits Per Value is as low as 9.03 for Xor8+ filter.

diff --git a/xorfilter.go b/xorfilter.go
index fae24a1..960f1ec 100644
--- a/xorfilter.go
+++ b/xorfilter.go
@@ -75 +75 @@ func (filter *Xor8) Contains(key uint64) bool {
-       h1 := h0 ^ (reduce(r1, 63) + 1)
+       h1 := (h0 + 1 + reduce(r1, filter.BlockLength-1)) % filter.BlockLength
@@ -90 +90 @@ func (filter *Xor8) geth0h1h2(k uint64) hashes {
-       answer.h1 = answer.h0 ^ (reduce(r1, 63) + 1)
+       answer.h1 = (answer.h0 + 1 + reduce(r1, filter.BlockLength-1)) % filter.BlockLength
@@ -102 +102 @@ func (filter *Xor8) geth1(h0 uint32, hash uint64) uint32 {
-       return h0 ^ (reduce(r1, 63) + 1)
+       return (h0 + 1 + reduce(r1, filter.BlockLength-1)) % filter.BlockLength
@lemire

This comment has been minimized.

Copy link
Collaborator

lemire commented Dec 26, 2019

Looks promising!!!

@lemire

This comment has been minimized.

Copy link
Collaborator

lemire commented Jan 6, 2020

I have merged it manually, with credit.

#14

Note that there is no reason to believe that this would work in general. That is, there is no analysis showing that we have a 50% probability of building the filter in general. Still, I decided to include it for now because it may prove useful in the future.

@lemire lemire closed this Jan 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.