Rust Metro64 and Metro128 bindings for kdb+
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.cargo
src
.gitignore
Cargo.lock
Cargo.toml
LICENSE
README.md
build.rs
krs-hash-dict-bench-func.q
krs-hash-dict-bench.q
krs-hash-str-bench.q
krs-hash-unit.q
krs-hash.q

README.md

kdb-rs-hash

Implement a Rust binding for hashing using Metro hash with CRC intrinsics supporting > 10GB/s hash rates on a single modern i5/i7 x86 core.

Review the write up at Rust, meet Q.

Build & Run

Note that this library requires a nightly rust and as been built on rust version 1.30.0-nightly (33b923fd4 2018-08-18). If you see errors related to to_ne_bytes not being defined or the like, you need to rustup to the currently nightly.

# create ./target/release/libkrshash.so 
$ cargo build --release

# NOTE: place libkrshash.so in your library path or next to your q binary
$ q krs-hash.q
KDB+ 3.6 2018.06.14 Copyright (C) 1993-2018 Kx Systems
m64/ 12()core 16384MB

q)rmetro128 `hello
fec3012b-5c96-d2f1-f8dd-32a0b77bfbe5
q)rmetro128 "hello"
fec3012b-5c96-d2f1-f8dd-32a0b77bfbe5

Measure performance using in memory and persisted data

$ q krs-hash-dict-bench.q 

OS-X notes

The target build a dylib that is 'weak linked', i.e. the kdb+ symbols are left undefined as the library is meant for inclusion in a running Q process.

For inclusion into the Q process, this can be renamed to .so. e.g.

$ cargo build --release && /bin/cp -rf ./target/release/libkrshash.dylib /opt/q/m64/libkrshash.so
$ q krs-hash.q

Benchmarking files

  • krs-hash.q: Includes Rust MetroHash bindings.
  • krs-hash-unit.q: q unit tests for binding.
  • krs-hash-str-bench.q: Benchmark test for hashes on strings.
  • krs-hash-dict-bench.q: Benchmark test for in-memory and mapped hashes on dictionaries.
  • krs-hash-dict-bench-func.q: q functions to compute benchmark test results.