perf(eval): incremental indexes for single-rule add paths

[mostafa]

Summary

Engine::add_rule and Engine::add_compiled_rule were rebuilding the inverted index and per-field bloom filter from scratch on every call, which made a tight loop of single-rule adds O(N²) in the total rule count. This PR makes both paths amortized O(1) by folding new rules into the indexes incrementally and only re-running the full bloom rebuild when a doubling watermark trips.

Batched loads (add_rules, extend_compiled_rules, add_collection) keep their existing one-rebuild-at-end fast path: for batches, a single full rebuild over the aggregate is still cheaper than N incremental appends.

Changes

rsigma-eval/src/rule_index.rs

• New RuleIndex::append_rule(rule_idx, rule) that folds a single rule's (field, exact_value) pairs into the index in time proportional to the rule's detection tree size, independent of the total rule count.
• RuleIndex::build becomes a thin wrapper that calls append_rule in order, then trusts rules.len() as the authoritative rule_count for the edge case where the tail of the slice contributes no pairs.

rsigma-eval/src/engine/bloom_index.rs

• New FieldBloomIndex::append_rule(rule) that inserts the rule's positive substring trigrams into existing per-field blooms and creates a tightly sized fresh bloom for fields that did not have one yet. No false negatives. FPR can drift upward between rebuilds, capped by the watermark below.
• New FieldBloomIndex::should_rebuild(rule_count) returning whether the rule count has at least doubled since the last full rebuild (with a 64-rule floor). This schedule keeps per-rule cost amortized O(1) while preventing FPR drift from running away.
• New rebuild_baseline field tracking the rule count at the most recent full rebuild; reset on build / build_with_budget.
• Extracted count_unique_trigrams helper shared by the batched and incremental paths.

rsigma-eval/src/engine/mod.rs

• New private Engine::index_append_last_rule that folds the just-pushed rule into both indexes incrementally, then triggers a full bloom rebuild via the existing build / build_with_budget path when the watermark trips. Cross-rule AC (daachorse) has no incremental update story, so when cross_rule_ac_enabled is on this method falls back to Engine::rebuild_index.
• Engine::add_rule and Engine::add_compiled_rule now call index_append_last_rule instead of rebuild_index. Behaviour preserved across all engine entry points.

rsigma-eval/README.md

• API table notes the new amortized O(1) characteristic of add_rule and add_compiled_rule.

Correctness

Probe semantics are unchanged. Between bloom rebuilds the incremental index can answer MaybeMatch where the batched-rebuild path would answer DefinitelyNoMatch, but both verdicts are safe: MaybeMatch just means the engine evaluates the rule directly instead of short-circuiting. There are no false negatives.

Tests

• RuleIndex::tests::test_append_rule_matches_build and test_append_rule_grows_rule_count pin equivalence between batched and incremental paths plus the monotonic rule_count contract.
• bloom_index::tests::append_rule_matches_build_verdicts confirms positive verdicts match across both paths and disjoint haystacks remain rejected under batched and at worst conservative (MaybeMatch) under incremental.
• bloom_index::tests::append_rule_creates_filter_for_new_field verifies a new field's trigrams are probable immediately after the first append.
• bloom_index::tests::should_rebuild_follows_doubling_watermark pins the 64-rule floor and 2x doubling schedule.
• New engine::tests::test_add_rule_loop_scales_linearly_on_large_corpus mirrors the existing batched-load scaling guard for the single-rule entry point: 2000 consecutive add_rule calls must complete inside a 30s ceiling.

Test plan

• cargo fmt --all -- --check
• cargo clippy --workspace --all-targets --all-features -- -D warnings
• cargo test --workspace --all-features (442 in rsigma-eval lib, full workspace green)
• rsigma validate ../sigma/rules/ against the full SigmaHQ corpus (3120 rules) in release: 1.0s wall.

References

• perf(eval): batch rule loading to avoid O(N²) index rebuilds #119