Skip to content

feat(tdigest): Add owned batch merging - #276

Open
ryux1 wants to merge 1 commit into
apache:mainfrom
ryux1:ryux1/feat/tdigest-owned-batch-merge
Open

feat(tdigest): Add owned batch merging#276
ryux1 wants to merge 1 commit into
apache:mainfrom
ryux1:ryux1/feat/tdigest-owned-batch-merge

Conversation

@ryux1

@ryux1 ryux1 commented Sep 6, 2026

Copy link
Copy Markdown

Closes #269.

This is the focused owned-batch-merging half of #261, derived from @tisonkun's original implementation. The batch quantile API is intentionally excluded and tracked separately in #275.

This change:

  • implements FromIterator with an owned k-way centroid merge
  • preserves a single non-empty input without recompression and ignores empty inputs
  • uses the smallest k across inputs and checks total-weight overflow
  • materializes and stably sorts only inputs with uncompressed tails
  • adds coverage for mixed k, empty/single inputs, compressed and uncompressed inputs, stable tie ordering, and overflow
  • adds comparison benchmarks for repeated borrowed merging and consuming collection

Validation:

  • cargo +nightly fmt --all --check
  • cargo x check
  • cargo x lint
  • cargo x test
  • cargo test --package tests-integration --test tdigest_test (26 passed)

Benchmark (100 samples, 64 partial digests):

  • repeated merge median: 205.1 us, 53 allocations / 234.5 KB total
  • FromIterator median: 58.23 us, 2 allocations / 131 KB total
  • uncompressed FromIterator median: 156.3 us, 2 allocations / 131 KB total

Co-authored-by attribution to @tisonkun is preserved in the commit.

Co-authored-by: tison <wander4096@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add owned batch merging to T-Digest

1 participant