Skip to content

DEV-1707 Stage 3: unified reroot_aggregate_key (source + args + kwargs + column_filter) - #266

Merged
ZmeiGorynych merged 3 commits into
egor/dev-1703-comprehensive-approach-expressions-crossing-joins-on-thefrom
egor/dev-1707-dev-1703-stage-3-unified-reroot_aggregate_key-source-args
Aug 2, 2026
Merged

DEV-1707 Stage 3: unified reroot_aggregate_key (source + args + kwargs + column_filter)#266
ZmeiGorynych merged 3 commits into
egor/dev-1703-comprehensive-approach-expressions-crossing-joins-on-thefrom
egor/dev-1707-dev-1703-stage-3-unified-reroot_aggregate_key-source-args

Conversation

@ZmeiGorynych

@ZmeiGorynych ZmeiGorynych commented Aug 2, 2026

Copy link
Copy Markdown
Member

DEV-1703 Stage 3. Closes DEV-1707; closes the cross-model residuals of DEV-1476 (c) and (d-cross).

What

One pure reroot_aggregate_key(key, *, target_path) in slayer/core/keys.py re-anchors all embedded references of an AggregateKey — source, positional args, kwarg values, and column_filter_key — symmetrically when a cross-model aggregate renders in its target scope. It replaces the three scattered per-field strip implementations that had diverged into two semantics:

  • slayer/engine/cross_model_planner.py::_local_agg_formula / _reroot_col_kwarg (string-level, prefix-strip-with-residual)
  • slayer/sql/generator.py::_render_cross_model_cte source rebuild + _reroot_kwarg + local_args (key-level, exact-match)
  • slayer/sql/generator.py HAVING route _reroot_having (near-verbatim copy of the above)

Unified on prefix-strip-with-residual (('customers','regions') under target ('customers',)('regions',); exact match → local). The generator's old exact-match is subsumed. Non-matching paths pass through unchanged — the function is total and never raises; a genuinely mis-pathed ref still surfaces at the downstream binder / kwarg-path validator exactly as before.

column_filter_key is owner-anchored (its canonical_sql / referenced_join_paths are relative to the model owning the filtered column), hence invariant under reroot and copied through unchanged. A rerooted filtered cross-model aggregate still reads local-source + non-empty filter paths — the DEV-1503 filtered-local trigger shape.

Closes

  • DEV-1476 (c): positional args now strip the target prefix in lockstep with kwargs.
  • DEV-1476 (d-cross): a path-bearing ColumnSqlKey explicit time arg is rerooted to a target-local key before _resolve_explicit_time_col, so it no longer raises.

The _resolve_explicit_time_col residual-hop guard is reworded from the now-closed DEV-1476 to DEV-1526 / Stage 4 (the isolated CTE does not yet pull a join a hop past the target). The analogous bare-column residual case is caught loudly by the SLAYER_VALIDATE_SCOPES scope-closure validator.

Out of scope

DEV-1476 (b) (no-time-dimension gating — Stage 6); CTE ScopeFrame migration (Stage 4).

Tests

New tests/test_reroot_aggregate_key.py (45 tests) pins the pure key algebra (source/args/kwargs × exact/deeper-residual/non-matching/scalar for both ColumnKey and ColumnSqlKey), column_filter_key invariance + the DEV-1503 trigger shape, the _local_agg_formula behaviour-lock, empty-target identity, and the reworded DEV-1526 guard. Codex reviewed both the plan and the tests; findings folded in.

Full non-integration suite green (8640 passed, 48 skipped, 75 xfailed, 1 unrelated xpassed); ruff check slayer/ tests/ clean. DECISIONS.md + docs/architecture/cross-model-aggregates.md updated.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Improved cross-model aggregate handling by resolving sources, arguments, keyword values, and column references in the correct query scope.
    • Preserved residual join paths and supported scalar and wildcard aggregate arguments.
  • Bug Fixes

    • Prevented inconsistent aggregate references during query planning and SQL generation.
    • Added clearer validation for unsupported deeper time-argument paths.
  • Documentation

    • Documented cross-model aggregate behavior and its limitations.
  • Tests

    • Added comprehensive coverage for rerooting, path handling, immutability, filtering, and aggregate rendering.

ZmeiGorynych and others added 2 commits August 2, 2026 17:14
…s + column_filter)

One pure `reroot_aggregate_key(key, *, target_path)` in slayer/core/keys.py
re-anchors ALL embedded references of an AggregateKey — source, positional
args, kwarg values, and column_filter_key — symmetrically when a cross-model
aggregate renders in its target scope. Replaces the three scattered per-field
strip implementations that had diverged into two semantics:

- cross_model_planner.py::_local_agg_formula / _reroot_col_kwarg (string-level)
- generator.py::_render_cross_model_cte source/_reroot_kwarg/local_args block
- generator.py HAVING-route _reroot_having block

Unified on prefix-strip-with-residual (planner semantics; the generator's old
exact-match is subsumed). Non-matching paths pass through unchanged (the
function is total, never raises). column_filter_key is owner-anchored, hence
invariant under reroot and copied through unchanged.

Closes DEV-1476 (c): positional args now strip the target prefix in lockstep
with kwargs. Closes DEV-1476 (d-cross): a path-bearing ColumnSqlKey explicit
time arg is rerooted to a target-local key before _resolve_explicit_time_col,
so it no longer raises. The residual deeper-hop guard is reworded from the
now-closed DEV-1476 to DEV-1526/Stage 4 (the isolated CTE does not yet pull
the deeper join); the analogous bare-column case is caught by the
SLAYER_VALIDATE_SCOPES scope-closure validator.

Full non-integration suite green; new tests/test_reroot_aggregate_key.py
(45 tests) pins the pure key algebra, filter-key invariance, the
_local_agg_formula behaviour-lock, and the reworded guard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…oach-expressions-crossing-joins-on-the' into egor/dev-1707-dev-1703-stage-3-unified-reroot_aggregate_key-source-args

# Conflicts:
#	DECISIONS.md
@linear

linear Bot commented Aug 2, 2026

Copy link
Copy Markdown

DEV-1707

DEV-1476

DEV-1703

@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: bb5676bb-1aae-4e76-883c-e8385dc67f39

📥 Commits

Reviewing files that changed from the base of the PR and between bcaf249 and 851cea4.

📒 Files selected for processing (2)
  • slayer/core/keys.py
  • tests/test_reroot_aggregate_key.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • slayer/core/keys.py

📝 Walkthrough

Walkthrough

Adds reroot_aggregate_key to unify cross-model aggregate path handling. Planner and SQL generator integrations reroot sources, arguments, and keyword values consistently. Tests cover residual paths, metadata preservation, immutability, rendering, and explicit time-argument failures.

Changes

Cross-model aggregate rerooting

Layer / File(s) Summary
Aggregate-key rerooting contract
slayer/core/keys.py, tests/test_reroot_aggregate_key.py, docs/architecture/cross-model-aggregates.md, DECISIONS.md
Adds shared rerooting for aggregate sources and embedded arguments. Matching prefixes are removed, residual paths are preserved, nonmatching values pass through, and column_filter_key remains unchanged.
Planner formula integration
slayer/engine/cross_model_planner.py, tests/test_reroot_aggregate_key.py
Applies shared rerooting to local aggregate formulas and preserves rendering for references and scalar arguments.
SQL rendering integration and guards
slayer/sql/generator.py, tests/test_reroot_aggregate_key.py
Applies rerooting to cross-model CTE and routed HAVING aggregates. Residual first/last time paths report DEV-1526 failures.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CrossModelPlanner
  participant reroot_aggregate_key
  participant SQLGenerator
  participant AggregateKey
  CrossModelPlanner->>reroot_aggregate_key: reroot source and aggregate arguments
  reroot_aggregate_key-->>CrossModelPlanner: return target-local AggregateKey
  SQLGenerator->>reroot_aggregate_key: reroot CTE or HAVING aggregate
  reroot_aggregate_key-->>SQLGenerator: return normalized AggregateKey
  SQLGenerator->>AggregateKey: render residual references and preserved filter metadata
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the DEV-1707 stage and the unified reroot_aggregate_key change across sources, arguments, kwargs, and column filters.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch egor/dev-1707-dev-1703-stage-3-unified-reroot_aggregate_key-source-args

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
slayer/sql/generator.py (1)

4436-4463: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Fail fast for residual ColumnKey time arguments. SLAYER_VALIDATE_SCOPES is opt-in and is enabled only by the test suite. A residual ColumnKey can therefore emit an unjoined alias such as customers__regions.population; add a symmetric eager NotImplementedError in the ColumnKey branch.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@slayer/sql/generator.py` around lines 4436 - 4463, The ColumnKey branch in
the first/last positional-argument handling must fail fast when a residual join
path exists, rather than constructing an unjoined relation alias. Add a
symmetric NotImplementedError for ColumnKey values with a non-empty path, while
preserving the current relation/leaf return for pathless keys and the existing
ColumnSqlKey behavior.
🧹 Nitpick comments (2)
slayer/core/keys.py (1)

518-529: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider rebuilding with model_copy to avoid field drift.

The function enumerates all five AggregateKey fields explicitly. That matches the current model. If a field is added to AggregateKey later, reroot silently drops it from the returned key, and the identity changes without any test failure in this module. model_copy(update=...) re-anchors only the fields that reroot owns.

Note that model_copy skips the _canonicalize_kwargs validator. Rerooting preserves kwarg names, so the input order is already canonical and the sort is a no-op. test_kwargs_canonical_sort_preserved_after_reroot pins that behavior either way.

♻️ Proposed refactor
-    return AggregateKey(
-        source=_reroot_path_ref(key.source, target_path=target_path),
-        agg=key.agg,
-        args=tuple(
-            _reroot_path_ref(a, target_path=target_path) for a in key.args
-        ),
-        kwargs=tuple(
-            (k, _reroot_path_ref(v, target_path=target_path))
-            for k, v in key.kwargs
-        ),
-        column_filter_key=key.column_filter_key,
-    )
+    return key.model_copy(update={
+        "source": _reroot_path_ref(key.source, target_path=target_path),
+        "args": tuple(
+            _reroot_path_ref(a, target_path=target_path) for a in key.args
+        ),
+        "kwargs": tuple(
+            (k, _reroot_path_ref(v, target_path=target_path))
+            for k, v in key.kwargs
+        ),
+    })
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@slayer/core/keys.py` around lines 518 - 529, Update the AggregateKey reroot
construction to use model_copy(update=...) on the existing key, replacing only
source, args, and kwargs with their rerooted values while preserving all other
fields automatically. Keep kwarg ordering unchanged, relying on the existing
canonical input order.
tests/test_reroot_aggregate_key.py (1)

570-576: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Hoist the fixture call out of the pytest.raises block.

_guard_source_model() runs inside the with pytest.raises(NotImplementedError) block. If that helper ever raises, the test passes for the wrong reason. This is also what SonarCloud flags at Line 570.

♻️ Proposed refactor
+    source_model = _guard_source_model()
     with pytest.raises(NotImplementedError) as excinfo:
         gen._resolve_explicit_time_col(
             key=key,
-            source_model=_guard_source_model(),
+            source_model=source_model,
             source_relation="customers",
             bundle=None,
         )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_reroot_aggregate_key.py` around lines 570 - 576, Move the
_guard_source_model() call before the pytest.raises(NotImplementedError) context
and store its result, then pass that stored source model to
gen._resolve_explicit_time_col. Keep only the intended method call inside the
exception assertion so the test validates the correct failure.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@slayer/sql/generator.py`:
- Around line 4436-4463: The ColumnKey branch in the first/last
positional-argument handling must fail fast when a residual join path exists,
rather than constructing an unjoined relation alias. Add a symmetric
NotImplementedError for ColumnKey values with a non-empty path, while preserving
the current relation/leaf return for pathless keys and the existing ColumnSqlKey
behavior.

---

Nitpick comments:
In `@slayer/core/keys.py`:
- Around line 518-529: Update the AggregateKey reroot construction to use
model_copy(update=...) on the existing key, replacing only source, args, and
kwargs with their rerooted values while preserving all other fields
automatically. Keep kwarg ordering unchanged, relying on the existing canonical
input order.

In `@tests/test_reroot_aggregate_key.py`:
- Around line 570-576: Move the _guard_source_model() call before the
pytest.raises(NotImplementedError) context and store its result, then pass that
stored source model to gen._resolve_explicit_time_col. Keep only the intended
method call inside the exception assertion so the test validates the correct
failure.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c9031c18-36dd-498a-ace2-86de1aeaabdc

📥 Commits

Reviewing files that changed from the base of the PR and between 66c64fa and bcaf249.

📒 Files selected for processing (6)
  • DECISIONS.md
  • docs/architecture/cross-model-aggregates.md
  • slayer/core/keys.py
  • slayer/engine/cross_model_planner.py
  • slayer/sql/generator.py
  • tests/test_reroot_aggregate_key.py

- tests/test_reroot_aggregate_key.py: hoist _guard_source_model() out of the
  pytest.raises block so the only possibly-throwing call inside it is the one
  under test (Sonar python:S5778 MAJOR; CodeRabbit tests nitpick).
- slayer/core/keys.py: rebuild the rerooted AggregateKey via model_copy(update=)
  so fields reroot doesn't own (agg, column_filter_key, any future field) ride
  through instead of being enumerated and silently droppable (CodeRabbit nitpick).
  Output-equivalent — model_copy skips _canonicalize_kwargs but reroot preserves
  kwarg names/order so the already-canonical sort is unchanged.

Not applied: CodeRabbit's "fail fast for residual ColumnKey time args" in
_resolve_explicit_time_col — the suggested guard would break the green
test_composite_first_last_with_joined_time_arg_adds_join (a LOCAL aggregate
legally uses a joined-path ColumnKey time arg the base SELECT pulls); the
method cannot distinguish that from the cross-model-CTE residual case, so the
guard belongs at the CTE call site and is DEV-1526/Stage-4 scope.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@sonarqubecloud

sonarqubecloud Bot commented Aug 2, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant