Implement a Rust binding for hashing using Metro hash with CRC intrinsics supporting > 10GB/s hash rates on a single modern i5/i7 x86 core.
Review the write up at Rust, meet Q.
Build & Run
Note that this library requires a nightly rust and as been built on
rust version 1.30.0-nightly (33b923fd4 2018-08-18). If you see errors related to
to_ne_bytes not being defined or the like, you need to rustup to the currently nightly.
# create ./target/release/libkrshash.so $ cargo build --release # NOTE: place libkrshash.so in your library path or next to your q binary $ q krs-hash.q KDB+ 3.6 2018.06.14 Copyright (C) 1993-2018 Kx Systems m64/ 12()core 16384MB q)rmetro128 `hello fec3012b-5c96-d2f1-f8dd-32a0b77bfbe5 q)rmetro128 "hello" fec3012b-5c96-d2f1-f8dd-32a0b77bfbe5
Measure performance using in memory and persisted data
$ q krs-hash-dict-bench.q
The target build a dylib that is 'weak linked', i.e. the kdb+ symbols are left undefined as the library is meant for inclusion in a running Q process.
For inclusion into the Q process, this can be renamed to
$ cargo build --release && /bin/cp -rf ./target/release/libkrshash.dylib /opt/q/m64/libkrshash.so $ q krs-hash.q
krs-hash.q: Includes Rust MetroHash bindings.
krs-hash-unit.q: q unit tests for binding.
krs-hash-str-bench.q: Benchmark test for hashes on strings.
krs-hash-dict-bench.q: Benchmark test for in-memory and mapped hashes on dictionaries.
krs-hash-dict-bench-func.q: q functions to compute benchmark test results.