planner/optimizer — `translator.rs::retention_count_for_assignment` (around line 60) hardcodes `1` for `QueryMethod::Subtract`, while `candidate_gen.rs` sets the deployed config's actual retained-window count to `n_windows`. Self-documented as a Phase-3 placeholder, but it's a silent mismatch waiting to bite once wired up.
Separately: for `Subtract`, retaining 1 aggregate makes sense for the current window, but we also need to retain another aggregate representing the sketch used for the previous query instance (to subtract against). It's possible this single calculation/parameter is being used to mean two different things and needs to be split into two distinct parameters.
Needs a design discussion to disambiguate before fixing.
Found during code review of PR #407.
Resolution
Traced every consumer of both fields before changing anything:
AggregationConfig.num_aggregates_to_retain (set by candidate_gen.rs to n_windows for Subtract) is the engine-side retention cap — consumed for real by simple_map_store::per_key.rs/global.rs as the eviction cap on the checkpoint ring buffer. This was already correct: the engine needs n_windows depth so the checkpoint from range_a ago is still around when "now" arrives. No bug here.
AggregationReference.num_aggregates_to_retain (hardcoded 1 in translator.rs) is the query-time combine count — how many retained checkpoints a query actually reads to compute its answer. For Subtract this is a fixed constant, 2 (current cumulative checkpoint + the one from range_a ago), independent of n_windows/range_a/W — that's what makes Subtract O(1). The doc comment on QueryMethod::Subtract already said this; the 1 was just a stale placeholder.
- These are two different concepts that happen to share a field name across two different structs (
AggregationConfig vs AggregationReference), not one parameter doing double duty. No type/field split needed.
- Confirmed nothing currently executes Subtract's actual merge math at query time (
window_merger.rs's IncrementalMerger for subtractable accumulators is still a documented future). So no consumer dictates a richer shape (e.g. an offset/stride field) yet — adding one now would be designing for a consumer that doesn't exist. Deferred until real Subtract execution lands and tells us what it needs.
Fix applied: translator.rs::retention_count_for_assignment now returns 2 for QueryMethod::Subtract instead of the placeholder 1.
Considered also renaming AggregationReference.num_aggregates_to_retain to something like num_aggregates_to_combine for clarity, since it means something different from AggregationConfig's field of the same name. Deferred for now — it has a real consumer (streaming_config.rs::from_yaml_data, used by the legacy generator.rs pipeline via planner_output.rs) keyed on that exact field/wire name, so the rename would need to either preserve the serde wire key or update that reader too. Out of scope for this fix.
planner/optimizer — `translator.rs::retention_count_for_assignment` (around line 60) hardcodes `1` for `QueryMethod::Subtract`, while `candidate_gen.rs` sets the deployed config's actual retained-window count to `n_windows`. Self-documented as a Phase-3 placeholder, but it's a silent mismatch waiting to bite once wired up.
Separately: for `Subtract`, retaining 1 aggregate makes sense for the current window, but we also need to retain another aggregate representing the sketch used for the previous query instance (to subtract against). It's possible this single calculation/parameter is being used to mean two different things and needs to be split into two distinct parameters.
Needs a design discussion to disambiguate before fixing.
Found during code review of PR #407.
Resolution
Traced every consumer of both fields before changing anything:
AggregationConfig.num_aggregates_to_retain(set bycandidate_gen.rston_windowsfor Subtract) is the engine-side retention cap — consumed for real bysimple_map_store::per_key.rs/global.rsas the eviction cap on the checkpoint ring buffer. This was already correct: the engine needsn_windowsdepth so the checkpoint fromrange_aago is still around when "now" arrives. No bug here.AggregationReference.num_aggregates_to_retain(hardcoded1intranslator.rs) is the query-time combine count — how many retained checkpoints a query actually reads to compute its answer. For Subtract this is a fixed constant, 2 (current cumulative checkpoint + the one fromrange_aago), independent ofn_windows/range_a/W— that's what makes Subtract O(1). The doc comment onQueryMethod::Subtractalready said this; the1was just a stale placeholder.AggregationConfigvsAggregationReference), not one parameter doing double duty. No type/field split needed.window_merger.rs'sIncrementalMergerfor subtractable accumulators is still a documented future). So no consumer dictates a richer shape (e.g. an offset/stride field) yet — adding one now would be designing for a consumer that doesn't exist. Deferred until real Subtract execution lands and tells us what it needs.Fix applied:
translator.rs::retention_count_for_assignmentnow returns2forQueryMethod::Subtractinstead of the placeholder1.Considered also renaming
AggregationReference.num_aggregates_to_retainto something likenum_aggregates_to_combinefor clarity, since it means something different fromAggregationConfig's field of the same name. Deferred for now — it has a real consumer (streaming_config.rs::from_yaml_data, used by the legacygenerator.rspipeline viaplanner_output.rs) keyed on that exact field/wire name, so the rename would need to either preserve the serde wire key or update that reader too. Out of scope for this fix.