perf(hashes): SIMD-batched SipHash-2-4 for BIP158 filter matching#845
Conversation
|
Warning Review limit reached
Next review available in: 35 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (8)
📝 WalkthroughWalkthroughThis PR adds a batched SipHash24 implementation and switches compact-filter matching to a pre-bucketed ChangesSipHash Core and Batching
FilterQuery-Based Compact Filter Matching
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## dev #845 +/- ##
==========================================
+ Coverage 73.34% 73.43% +0.08%
==========================================
Files 324 324
Lines 72923 73150 +227
==========================================
+ Hits 53486 53715 +229
+ Misses 19437 19435 -2
|
ZocoLini
force-pushed
the
perf/siphash
branch
4 times, most recently
from
f1c14f5 to
9747d1f
Compare
|
@CodeRabbit review |
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
dash/src/bip158.rs (1)
392-399: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
pushusesexpectin library code.The
try_into().expect(...)calls are provably infallible here (guarded by thelen()match), but per coding guidelines library code should avoidexpect(). Since the invariant is local to thismatch, the risk is only that a future edit to the arm/length constants silently reintroduces a panic. Optional to harden.As per coding guidelines: "Avoid
unwrap()andexpect()in library code; use proper error types".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@dash/src/bip158.rs` around lines 392 - 399, The Bip158::push method uses expect in library code for the 25-byte and 23-byte arms, which should be avoided even if the match currently makes the conversion infallible. Replace the try_into().expect(...) calls with a non-panicking conversion path in push, or otherwise make the length-to-type routing explicit so the invariant is enforced without expect. Keep the fix localized to push and the p2pkh/p2sh handling so future changes to the length buckets do not reintroduce a panic.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@hashes/src/siphash24.rs`:
- Around line 334-366: The `siphash_batch` public API only validates `out.len()
>= inputs.len()` with `debug_assert!`, so release builds can reach the unsafe
AVX2/NEON paths and write past the end of `out`. Replace that check in
`siphash_batch` with a runtime `assert!` (or equivalent always-on validation)
before calling `hash_many_wide_avx2`, `hash_many_neon_wide`, or `siphash_each`,
so undersized output buffers fail deterministically in all build modes.
---
Nitpick comments:
In `@dash/src/bip158.rs`:
- Around line 392-399: The Bip158::push method uses expect in library code for
the 25-byte and 23-byte arms, which should be avoided even if the match
currently makes the conversion infallible. Replace the try_into().expect(...)
calls with a non-panicking conversion path in push, or otherwise make the
length-to-type routing explicit so the invariant is enforced without expect.
Keep the fix localized to push and the p2pkh/p2sh handling so future changes to
the length buckets do not reintroduce a panic.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 6f1ebe5f-249f-4797-ae34-e8e52917a35a
📒 Files selected for processing (8)
dash-spv/src/sync/filters/manager.rsdash/src/bip152.rsdash/src/bip158.rshashes/Cargo.tomlhashes/benches/hashes.rshashes/benches/siphash.rshashes/src/siphash24.rskey-wallet-manager/src/matching.rs
xdustinface
left a comment
xdustinface
left a comment
There was a problem hiding this comment.
Nice! Have a look at the one comment i posted.
2 participants
Adds siphash() (scalar one-shot) and siphash_batch::() (SIMD: AVX2 12-lane on x86_64, NEON 12-message on aarch64, with a scalar fallback), bit-identical to the reference and verified against the official test vectors on both architectures.
Wires it into BIP158 through FilterQuery (elements bucketed by length once and reused): both filter matching (match_any/match_all) and filter building (GcsFilterWriter) now batch-hash the whole element set instead of one element at a time. The hot path when a wallet scans block filters against its watched scripts.
Note on the siphash HashEngine:
The engine form of siphash (siphash24::HashEngine, its hash_with_keys / hash_to_u64_with_keys helpers, and the io::Write impl) is unused anywhere in this repo — every call site goes through the free siphash() / siphash_batch()
functions. It's dead internally, but it's public API and matches the uniform "one HashEngine per hash" pattern, so this PR leaves it in place (removing it would be a breaking change). Flagging it in case we later want to slim the crate's public surface.
Longer term, batching would be cleaner as a trait — a "hash N equal-length inputs" method that any hash engine can implement — instead of a standalone siphash_batch function. Callers could be generic over it, and future SIMD work (other hashes, or an AVX-512 path) would reuse shared machinery instead of being duplicated per algorithm. Out of scope here, just worth noting
Summary by CodeRabbit