Skip to content

Commit

Permalink
ARROW-6020: [Java] Refactor ByteFunctionHelper#hash with new added Ar…
Browse files Browse the repository at this point in the history
…rowBufHasher

Related to [ARROW-6020](https://issues.apache.org/jira/browse/ARROW-6020).
Some logic in these two classes are similar, should replace ByteFunctionHelper#hash logic with ArrowBufHasher since it has murmur hash algorithm which could avoid hash collision.

Closes #4938 from tianchen92/ARROW-6020 and squashes the following commits:

108ae18 <tianchen> fix style
8591766 <tianchen> ARROW-6020:  Refactor ByteFunctionHelper#hash with new added ArrowBufHasher

Authored-by: tianchen <niki.lj@alibaba-inc.com>
Signed-off-by: Micah Kornfield <emkornfield@gmail.com>
  • Loading branch information
tianchen92 authored and emkornfield committed Aug 1, 2019
1 parent 2ffb7f8 commit 0823b49
Showing 1 changed file with 4 additions and 27 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
package org.apache.arrow.memory.util;

import org.apache.arrow.memory.BoundsChecking;
import org.apache.arrow.memory.util.hash.ArrowBufHasher;
import org.apache.arrow.memory.util.hash.DirectHasher;

import io.netty.buffer.ArrowBuf;
import io.netty.util.internal.PlatformDependent;
Expand Down Expand Up @@ -250,35 +252,10 @@ private static int memcmp(
* Compute hashCode with the given {@link ArrowBuf} and start/end index.
*/
public static final int hash(final ArrowBuf buf, int start, int end) {
long addr = buf.memoryAddress();
int len = end - start;
long pos = addr + start;

int hash = 0;
ArrowBufHasher hasher = DirectHasher.INSTANCE;

while (len > 7) {
long value = PlatformDependent.getLong(pos);
hash = comebineHash(hash, Long.hashCode(value));

pos += 8;
len -= 8;
}

while (len > 3) {
int value = PlatformDependent.getInt(pos);
hash = comebineHash(hash, value);

pos += 4;
len -= 4;
}

while (len-- != 0) {
byte value = PlatformDependent.getByte(pos);
hash = comebineHash(hash, value);
pos ++;
}

return hash;
return hasher.hashCode(buf, start, end - start);
}

/**
Expand Down

0 comments on commit 0823b49

Please sign in to comment.