feat: adopt sketch-core runtime modules into existing sketches/ layout#36
Merged
feat: adopt sketch-core runtime modules into existing sketches/ layout#36
Conversation
zzylol
force-pushed
the
refactor/adopt-sketch-core-modules
branch
from
63051a8 to
e473ccc
Compare
zzylol
changed the title
zzylol
added a commit
to ProjectASAP/ASAPQuery
that referenced
this pull request
May 1, 2026
Update PR #307 against the reorganized asap_sketchlib (PR #36) — runtime sketches now live in the existing src/sketches/ layout (single home per sketch concept), not under a separate `asap::` namespace. Path swaps in asap-query-engine (legacy fork; smaller surface than ASAPQuery-backend — no DDSketch / CountSketch / HllSketch accumulators): - asap_sketchlib::asap::count_min::* → ::sketches::countmin::* - asap_sketchlib::asap::kll::* → ::sketches::kll::* - asap_sketchlib::asap::count_min_with_heap::* → ::sketches::cms_heap::* - asap_sketchlib::asap::hydra_kll::* → ::sketches::hydra_kll::* - asap_sketchlib::asap::set_aggregator::* → ::sketches::set_aggregator::* - asap_sketchlib::asap::delta_set_aggregator::* → ::sketches::delta_set_aggregator::* - asap_sketchlib::asap::config::* → ::asap_runtime::* Rename carried through: - HeapItem → CmsHeapItem (avoids common::input::HeapItem collision) main.rs aliases asap_runtime as `config` so existing clap-derive references keep working. Tests: - cargo check -p query_engine_rust → clean - cargo test -p query_engine_rust --lib precompute_operators → 87 passed, 0 failed Depends on ProjectASAP/asap_sketchlib#36 (force-pushed e473ccc). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
force-pushed
the
refactor/adopt-sketch-core-modules
branch
4 times, most recently
from
8db5e7f to
943817e
Compare
zzylol
added a commit
to ProjectASAP/ASAPQuery
that referenced
this pull request
May 1, 2026
Update PR #307 against the reorganized asap_sketchlib (PR #36) — runtime sketches now live in the existing src/sketches/ layout (single home per sketch concept), not under a separate `asap::` namespace. Path swaps in asap-query-engine (legacy fork; smaller surface than ASAPQuery-backend — no DDSketch / CountSketch / HllSketch accumulators): - asap_sketchlib::asap::count_min::* → ::sketches::countmin::* - asap_sketchlib::asap::kll::* → ::sketches::kll::* - asap_sketchlib::asap::count_min_with_heap::* → ::sketches::cms_heap::* - asap_sketchlib::asap::hydra_kll::* → ::sketches::hydra_kll::* - asap_sketchlib::asap::set_aggregator::* → ::sketches::set_aggregator::* - asap_sketchlib::asap::delta_set_aggregator::* → ::sketches::delta_set_aggregator::* - asap_sketchlib::asap::config::* → ::asap_runtime::* Rename carried through: - HeapItem → CmsHeapItem (avoids common::input::HeapItem collision) main.rs aliases asap_runtime as `config` so existing clap-derive references keep working. Tests: - cargo check -p query_engine_rust → clean - cargo test -p query_engine_rust --lib precompute_operators → 87 passed, 0 failed Depends on ProjectASAP/asap_sketchlib#36 (force-pushed e473ccc). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
force-pushed
the
refactor/adopt-sketch-core-modules
branch
2 times, most recently
from
d5015ed to
0338b3b
Compare
zzylol
added a commit
to ProjectASAP/ASAPQuery
that referenced
this pull request
May 1, 2026
Update PR #307 against the reorganized asap_sketchlib (PR #36) — runtime sketches now live in the existing src/sketches/ layout (single home per sketch concept), not under a separate `asap::` namespace. Path swaps in asap-query-engine (legacy fork; smaller surface than ASAPQuery-backend — no DDSketch / CountSketch / HllSketch accumulators): - asap_sketchlib::asap::count_min::* → ::sketches::countmin::* - asap_sketchlib::asap::kll::* → ::sketches::kll::* - asap_sketchlib::asap::count_min_with_heap::* → ::sketches::cms_heap::* - asap_sketchlib::asap::hydra_kll::* → ::sketches::hydra_kll::* - asap_sketchlib::asap::set_aggregator::* → ::sketches::set_aggregator::* - asap_sketchlib::asap::delta_set_aggregator::* → ::sketches::delta_set_aggregator::* - asap_sketchlib::asap::config::* → ::asap_runtime::* Rename carried through: - HeapItem → CmsHeapItem (avoids common::input::HeapItem collision) main.rs aliases asap_runtime as `config` so existing clap-derive references keep working. Tests: - cargo check -p query_engine_rust → clean - cargo test -p query_engine_rust --lib precompute_operators → 87 passed, 0 failed Depends on ProjectASAP/asap_sketchlib#36 (force-pushed e473ccc). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
force-pushed
the
refactor/adopt-sketch-core-modules
branch
from
0338b3b to
d84ff15
Compare
zzylol
added a commit
to ProjectASAP/ASAPQuery
that referenced
this pull request
May 1, 2026
Update PR #307 against the reorganized asap_sketchlib (PR #36) — runtime sketches now live in the existing src/sketches/ layout (single home per sketch concept), not under a separate `asap::` namespace. Path swaps in asap-query-engine (legacy fork; smaller surface than ASAPQuery-backend — no DDSketch / CountSketch / HllSketch accumulators): - asap_sketchlib::asap::count_min::* → ::sketches::countmin::* - asap_sketchlib::asap::kll::* → ::sketches::kll::* - asap_sketchlib::asap::count_min_with_heap::* → ::sketches::cms_heap::* - asap_sketchlib::asap::hydra_kll::* → ::sketches::hydra_kll::* - asap_sketchlib::asap::set_aggregator::* → ::sketches::set_aggregator::* - asap_sketchlib::asap::delta_set_aggregator::* → ::sketches::delta_set_aggregator::* - asap_sketchlib::asap::config::* → ::asap_runtime::* Rename carried through: - HeapItem → CmsHeapItem (avoids common::input::HeapItem collision) main.rs aliases asap_runtime as `config` so existing clap-derive references keep working. Tests: - cargo check -p query_engine_rust → clean - cargo test -p query_engine_rust --lib precompute_operators → 87 passed, 0 failed Depends on ProjectASAP/asap_sketchlib#36 (force-pushed e473ccc). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 0.5 of the edge-framework design doc (#213) — retire the standalone `sketch-core` crate and consolidate ASAP's runtime sketches into asap_sketchlib's existing `src/sketches/` layout (no separate `asap/` namespace). For each sketch type that already had a home in `src/sketches/`, the wire-format-aligned variant (with public bucket-array fields, sparse multi-cell delta types, `apply_delta`, msgpack ser/de, and `from_raw` constructors — the surface needed for proto decode + delta-transmission reconstitution; paper §6.2 B3 / B4) is appended to the existing file under a clearly-marked section header. The existing in-process variants (`DDSketch`, `CountMin<S, Mode, H>`, `Count`, `HyperLogLog`, `KLL<T>`, `CMSHeap`) are unchanged. Single-file homes (existing → +appended wire-format types): - `src/sketches/ddsketch.rs` +DdSketch + DdSketchDelta - `src/sketches/countmin.rs` +CountMinSketch + CountMinDelta - `src/sketches/count.rs` +CountSketch + CountSketchDelta - `src/sketches/hll.rs` +HllSketch + HllSketchDelta + HllVariant - `src/sketches/kll.rs` +KllSketch + KllSketchData + KllBackend - `src/sketches/cms_heap.rs` +CountMinSketchWithHeap + CmsHeapItem New sibling files (no existing home — follow the one-file-per-sketch convention): - `src/sketches/hydra_kll.rs` (HydraKllSketch) - `src/sketches/set_aggregator.rs` (SetAggregator) - `src/sketches/delta_set_aggregator.rs` - `src/sketches/count_min_sketchlib.rs` (FFI helper) - `src/sketches/count_min_with_heap_sketchlib.rs` (FFI helper) - `src/sketches/kll_sketchlib.rs` (FFI helper) Top-level: - `src/asap_runtime.rs` — `ImplMode` (Legacy | Sketchlib) backend selector, used by the runtime CountMin / KLL / CountMinWithHeap dispatch. Naming-conflict resolutions: - `octo_delta::HllDelta` already exists (single-register, octo path), so the wire-format multi-register delta is renamed to `HllSketchDelta`. - `common::input::HeapItem` already exists, so the wire-format CMSHeap item is renamed to `CmsHeapItem`. Other notes: - `SketchInput` references in the wrappers (sketch-core was pinned against an older asap_sketchlib) renamed to current `DataInput`; KLL update path switched from `&DataInput::F64(v)` to `&v` to match the typed `KLL<T>` generic API. - `clap::ValueEnum` derive on `ImplMode` is gated behind a new `asap-cli` feature so non-CLI consumers don't pay for `clap`. The query engine enables this feature. - New deps added to Cargo.toml: `xxhash-rust` (xxh32), `dsrs` (datasketches-rs), `clap` (optional, behind `asap-cli`), `ctor` (dev-dep for the legacy-mode test initializer that previously lived in sketch-core's lib.rs). Tests: cargo test --lib → 376 passed, 0 failed (308 existing + 68 ported wire-format tests). No regressions. Followup: ASAPQuery-backend PR #73 and ASAPQuery PR #307 will switch consumer imports from `sketch_core::*` and the older `asap_sketchlib::asap::*` to the new `asap_sketchlib::sketches::*` paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zzylol
force-pushed
the
refactor/adopt-sketch-core-modules
branch
from
d84ff15 to
5f990ae
Compare
zzylol
added a commit
to ProjectASAP/ASAPQuery
that referenced
this pull request
May 2, 2026
* refactor: retire sketch-core mirror * refactor: switch consumer imports to asap_sketchlib::sketches::* Update PR #307 against the reorganized asap_sketchlib (PR #36) — runtime sketches now live in the existing src/sketches/ layout (single home per sketch concept), not under a separate `asap::` namespace. Path swaps in asap-query-engine (legacy fork; smaller surface than ASAPQuery-backend — no DDSketch / CountSketch / HllSketch accumulators): - asap_sketchlib::asap::count_min::* → ::sketches::countmin::* - asap_sketchlib::asap::kll::* → ::sketches::kll::* - asap_sketchlib::asap::count_min_with_heap::* → ::sketches::cms_heap::* - asap_sketchlib::asap::hydra_kll::* → ::sketches::hydra_kll::* - asap_sketchlib::asap::set_aggregator::* → ::sketches::set_aggregator::* - asap_sketchlib::asap::delta_set_aggregator::* → ::sketches::delta_set_aggregator::* - asap_sketchlib::asap::config::* → ::asap_runtime::* Rename carried through: - HeapItem → CmsHeapItem (avoids common::input::HeapItem collision) main.rs aliases asap_runtime as `config` so existing clap-derive references keep working. Tests: - cargo check -p query_engine_rust → clean - cargo test -p query_engine_rust --lib precompute_operators → 87 passed, 0 failed Depends on ProjectASAP/asap_sketchlib#36 (force-pushed e473ccc). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 2, 2026
zzylol
added a commit
that referenced
this pull request
May 5, 2026
PR #36 silently reverted PR #37's hh_keys field on CountSketchDelta when its merge landed. Restore on top of the post-#39 module names (src/sketches/countsketch.rs). Mirrors sketchlib-go's Delta.HHKeys / apply-delta heavy-hitter rebuild path. Unblocks ASAPQuery-backend's PR-#74 consumer to pin asap_sketchlib to main, which in turn unblocks ASAPCollector Phase 3 step 3 (backend consumes asap-precompute-rs). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 0.5 of the edge-framework design doc (#213) — retire the standalone
sketch-corecrate and consolidate ASAP's runtime sketches into asap_sketchlib's existingsrc/sketches/layout (no separateasap/namespace).For each sketch that already had a home in
src/sketches/, the wire-format-aligned variant (with public bucket-array fields, sparse multi-cell delta types,apply_delta, msgpack ser/de,from_raw) is appended to the existing file under a clear section header. The existing in-process variants (DDSketch,CountMin<S, Mode, H>,Count,HyperLogLog,KLL<T>,CMSHeap) are unchanged.Changes
Single-file homes (existing + appended wire-format types):
src/sketches/ddsketch.rsDdSketch,DdSketchDeltasrc/sketches/countmin.rsCountMinSketch,CountMinDeltasrc/sketches/count.rsCountSketch,CountSketchDeltasrc/sketches/hll.rsHllSketch,HllSketchDelta,HllVariantsrc/sketches/kll.rsKllSketch,KllSketchData,KllBackendsrc/sketches/cms_heap.rsCountMinSketchWithHeap,CmsHeapItemNew sibling files (no existing home):
hydra_kll.rs,set_aggregator.rs,delta_set_aggregator.rscount_min_sketchlib.rs,count_min_with_heap_sketchlib.rs,kll_sketchlib.rs(FFI helpers)Top-level:
src/asap_runtime.rs—ImplMode(Legacy | Sketchlib) selector for the runtime CountMin / KLL / CountMinWithHeap dispatch.Naming-conflict resolutions
octo_delta::HllDeltaalready exists (single-register, octo path), so the wire-format multi-register delta is renamed toHllSketchDelta.common::input::HeapItemalready exists, so the wire-format CMSHeap item is renamed toCmsHeapItem.Tests
```
cargo test --lib → 376 passed, 0 failed
```
(308 existing + 68 ported wire-format tests; no regressions.)
Followups