-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reproducible Hashes is 0.8? #132
Comments
I am in a similar boat. I've tried setting static HASH_KEY: usize = 12345;
lazy_static! {
static ref HASH_STATE: RandomState = {
set_random_source(NotRandomState).expect("Setting hash state to not-random");
RandomState::with_seed(HASH_KEY)
};
}
struct NotRandomState;
impl RandomSource for NotRandomState {
fn gen_hasher_seed(&self) -> usize {
HASH_KEY
}
}
fn hash_stuff(a: A, b: B) -> u64 {
let mut hasher = HASH_STATE.build_hasher();
a.hash(&mut hasher);
b.hash(&mut hasher);
hasher.finish().into()
} This seems to produce self-consistent hashes between builds on the same platform. However, on different platforms (e.g. aarch64) it produces different values. Is there an easier or more reliable method of ensuring consistencency? |
@domenukk - digging into the code a bit farther, it seems that using e.g.
However, I still don't get consistency across multiple platforms (and I suspect 0.7 is the same) |
with_seeds is the method you are looking for. |
This is now better documented in 0.8.1: |
Reduces total number of packages from 577 to 571 on building with: `cargo +nightly build --workspace --all-features` * ahash 0.7 -> 0.8 * Move `AHasher::new_with_keys` to `RandomState::with_seeds` given the recommendation from: aHash maintainer: tkaitchuck/aHash#132 (comment) * bindgen: 0.61 -> 0.63 * c2rust-bitfields: 0.3 -> 0.17 * criterion: 0.3 -> 0.4 * crossterm: 0.25 -> 0.26 * dynasmrt: 1.2 -> 2 * frida-gum/frida-gum-sys * frida-gum: 0.8.1 -> 0.10 * frida-gum-sys: 0.4.1 => 0.6 * Update underlying frida version from 15.2 -> 16.0 * goblin: 0.5.3 -> 0.6 * hashbrown: 0.12 -> 0.13 * nix: 0.25 -> 0.26 * The `addr` arg of `mmap` is now of type `Option<NonZeroUsize>` * The `length` arg of `mmap` is now of type `NonZeroUsize` * prometheus-client: 0.18.0 -> 0.19 * Do not box metrics * Gauges (a majority of the LibAFL metrics) are now i64 types so there is a small chance of overflow, with the u64 values that LibAFL tracks, but unlikely to be problematic. * Keep `exec_rate` as a floating point value * serial_test: 0.8 -> 1 * typed-builder: 0.10.0 -> 0.12 * windows: 0.42.0 -> 0.44
Reduces total number of packages from 577 to 571 on building with: `cargo +nightly build --workspace --all-features` * ahash 0.7 -> 0.8 * Move `AHasher::new_with_keys` to `RandomState::with_seeds` given the recommendation from: aHash maintainer: tkaitchuck/aHash#132 (comment) * bindgen: 0.61 -> 0.63 * c2rust-bitfields: 0.3 -> 0.17 * criterion: 0.3 -> 0.4 * crossterm: 0.25 -> 0.26 * dynasmrt: 1.2 -> 2 * goblin: 0.5.3 -> 0.6 * hashbrown: 0.12 -> 0.13 * nix: 0.25 -> 0.26 * The `addr` arg of `mmap` is now of type `Option<NonZeroUsize>` * The `length` arg of `mmap` is now of type `NonZeroUsize` * Requires updating implementers to update `nix` as well * prometheus-client: 0.18.0 -> 0.19 * Do not box metrics * Gauges (a majority of the LibAFL metrics) are now i64 types so there is a small chance of overflow, with the u64 values that LibAFL tracks, but unlikely to be problematic. * Keep `exec_rate` as a floating point value * serial_test: 0.8 -> 1 * typed-builder: 0.10.0 -> 0.12 * windows: 0.42.0 -> 0.44
Reduces total number of packages from 577 to 571 on building with: `cargo +nightly build --workspace --all-features` * ahash 0.7 -> 0.8 * Move `AHasher::new_with_keys` to `RandomState::with_seeds` given the recommendation from: aHash maintainer: tkaitchuck/aHash#132 (comment) * bindgen: 0.61 -> 0.63 * c2rust-bitfields: 0.3 -> 0.17 * criterion: 0.3 -> 0.4 * crossterm: 0.25 -> 0.26 * dynasmrt: 1.2 -> 2 * goblin: 0.5.3 -> 0.6 * hashbrown: 0.12 -> 0.13 * nix: 0.25 -> 0.26 * The `addr` arg of `mmap` is now of type `Option<NonZeroUsize>` * The `length` arg of `mmap` is now of type `NonZeroUsize` * Requires updating implementers to update `nix` as well * prometheus-client: 0.18.0 -> 0.19 * Do not box metrics * Gauges (a majority of the LibAFL metrics) are now i64 types so there is a small chance of overflow, with the u64 values that LibAFL tracks, but unlikely to be problematic. * Keep `exec_rate` as a floating point value * serial_test: 0.8 -> 1 * typed-builder: 0.10.0 -> 0.12 * windows: 0.42.0 -> 0.44 Co-authored-by: Dominik Maier <domenukk@gmail.com>
We are currently using ahash 0.7 to hash some data we store to disk, as fast as possible, and deduplicate them between different processes (that might be compiled differently, as well).
It doesn't matter if it's attackable, since we are the only users in this case.
So far we have used new_with_keys(key1: u128, key2: u128) -> AHasher with 0, 0 as keys, to create such a reproducible hashes.
However, starting in 0.8 this method is private.
Are we holding it wrong/misunderstanding something?
What would be the fastest way to create such a reproducible ahasher in 0.8 and up?
Even
RandomState::with_seed(0).build_hasher();
seems to internally use the keys.Thank you
The text was updated successfully, but these errors were encountered: