New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

codec/statistics: implement fmsketch #2286

Merged
merged 26 commits into from Sep 21, 2017

Conversation

Projects
None yet
5 participants
@AndreMouche
Member

AndreMouche commented Sep 12, 2017

@AndreMouche

This comment has been minimized.

Show comment
Hide comment
@AndreMouche
Member

AndreMouche commented Sep 14, 2017

PTAL

let hash = {
let mut out: [u8; 16] = [0; 16];
murmur3_x64_128(&mut bytes, 0, &mut out);
LittleEndian::read_u64(&out[0..8])

This comment has been minimized.

@BusyJay

BusyJay Sep 18, 2017

Contributor

Why initialized as 16 but only takes 8?

@BusyJay

BusyJay Sep 18, 2017

Contributor

Why initialized as 16 but only takes 8?

This comment has been minimized.

@lamxTyler

lamxTyler Sep 18, 2017

Contributor

Usually, the 64-bit version of murmur is done by drop the second half of the 128-bit version, so it will only takes 8.

@lamxTyler

lamxTyler Sep 18, 2017

Contributor

Usually, the 64-bit version of murmur is done by drop the second half of the 128-bit version, so it will only takes 8.

@AndreMouche

This comment has been minimized.

Show comment
Hide comment
@AndreMouche
Member

AndreMouche commented Sep 19, 2017

@AndreMouche

This comment has been minimized.

Show comment
Hide comment
@AndreMouche
Member

AndreMouche commented Sep 19, 2017

Show outdated Hide outdated src/coprocessor/statistics/fmsketch.rs
Show outdated Hide outdated src/coprocessor/statistics/fmsketch.rs
pub fn build_fmsketch(values: &[Datum], max_size: usize) -> Result<FMSketch> {
let mut s = FMSketch::new(max_size);
for value in values {
let bytes = try!(datum::encode_value(as_slice(value)));

This comment has been minimized.

@BusyJay

BusyJay Sep 19, 2017

Contributor

Why as_slice?

@BusyJay

BusyJay Sep 19, 2017

Contributor

Why as_slice?

This comment has been minimized.

@AndreMouche

AndreMouche Sep 20, 2017

Member

Since encode_value needs a &[Datum] while value is &Datum @BusyJay

@AndreMouche

AndreMouche Sep 20, 2017

Member

Since encode_value needs a &[Datum] while value is &Datum @BusyJay

Show outdated Hide outdated src/coprocessor/statistics/fmsketch.rs
Show outdated Hide outdated src/coprocessor/statistics/fmsketch.rs
Show outdated Hide outdated src/coprocessor/statistics/fmsketch.rs
FMSketch {
mask: 0,
max_size: max_size,
hash_set: HashSet::with_capacity(max_size + 1),

This comment has been minimized.

@BusyJay

BusyJay Sep 20, 2017

Contributor

Why add 1?

@BusyJay

BusyJay Sep 20, 2017

Contributor

Why add 1?

@BusyJay

This comment has been minimized.

Show comment
Hide comment
@BusyJay

BusyJay Sep 20, 2017

Contributor

LGTM

Contributor

BusyJay commented Sep 20, 2017

LGTM

@AndreMouche

This comment has been minimized.

Show comment
Hide comment
@AndreMouche

AndreMouche Sep 20, 2017

Member

/run-all-tests

Member

AndreMouche commented Sep 20, 2017

/run-all-tests

@AndreMouche AndreMouche merged commit 1504db9 into master Sep 21, 2017

3 checks passed

ci/circleci Your tests passed on CircleCI!
Details
jenkins-ci-tikv/build Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details

@AndreMouche AndreMouche deleted the shirly/fmsketch branch Sep 21, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment