Repository navigation
flowstats v0.1.2
Bug Fixes
ReservoirSampler: Fixed biased merge producing non-uniform samples
The merge() implementation treated each item in the other reservoir as a single observation, incrementing count by 1 per reservoir slot instead of accounting for the actual number of stream items each slot represents. This produced heavily biased samples when merging reservoirs that had processed different stream sizes.
The fix uses weighted random selection: each output slot is drawn from self with probability self.count / total_count and from other with probability other.count / total_count, correctly preserving uniform sampling over the combined stream. Empty and underfilled reservoirs are handled as special cases.
TDigest: Fixed quantile/rank interpolation producing inconsistent results
quantile_impl() and rank_impl() used different interpolation models — quantile used cumulative boundaries while rank used centroid midpoints — causing rank(quantile(q)) ≠ q roundtrip failures. Both methods now use a consistent piecewise linear model through midpoint anchors: (rank=0, value=min) → (rank=cumulative_i + weight_i/2, value=mean_i) → (rank=count, value=max).
TDigest: Fixed division-by-zero in interpolation with duplicate values
When consecutive centroids had equal means (common with duplicate-heavy data), denominators in interpolation fractions could be zero, producing NaN. All division points in quantile_impl() and rank_impl() are now guarded with if denom <= 0.0 { return safe_boundary; }.
TDigest: Fixed potential NaN from asin() input drift
The scale function computed asin(2*q - 1) without clamping. Floating-point arithmetic could push the argument slightly outside [-1, 1], producing NaN. The argument is now clamped: (2.0 * q - 1.0).clamp(-1.0, 1.0).
TDigest: Fixed merge rejecting effectively-equal compression parameters
Merge used f64::EPSILON (~2×10⁻¹⁶) to compare compression parameters, which rejected values that differed only due to serialization roundtrips. Now uses relative tolerance of 1×10⁻⁶.
Performance
TDigest: Eliminated clone-per-query overhead
quantile() and rank() previously cloned the entire TDigest and compressed the clone on every call when the buffer was non-empty. The internal structure now uses RefCell for lazy compression on &self query methods, while mutation methods (add, merge) use get_mut() for zero-overhead access.
Note: This makes TDigest Send but not Sync. For shared concurrent access, wrap in Arc<Mutex<_>> or Arc<RwLock<_>>.
Other Improvements
- TDigest: Added compression parameter validation —
TDigest::new()now panics on non-finite, zero, or negative compression values instead of silently producing broken digests. - TDigest: Made
push()private — users should useadd()from theQuantileSketchtrait; the redundantpush()method is no longer part of the public API. - TDigest: Improved
num_centroids()documentation — clarifies that the count excludes values still in the uncompressed buffer. - README: Fixed incorrect default feature flags — now matches
Cargo.toml(all algorithm families enabled by default). - README: Fixed
CountMinSketchexample —add()requires a count argument. - README: Added
TDigestthread-safety note to the Parallel Processing section.