Skip to content

Use the reserved __giql_ prefix for CLUSTER/MERGE synthesized identifiers — Closes #161#177

Merged
conradbzura merged 2 commits into
mainfrom
161-giql-prefix-cluster-merge-identifiers
Jul 6, 2026
Merged

Use the reserved __giql_ prefix for CLUSTER/MERGE synthesized identifiers — Closes #161#177
conradbzura merged 2 commits into
mainfrom
161-giql-prefix-cluster-merge-identifiers

Conversation

@conradbzura

@conradbzura conradbzura commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Move the three CLUSTER/MERGE synthesized SQL identifiers into the reserved __giql_ namespace so they cannot collide with user-supplied names. The per-row flag is_new_cluster, the CLUSTER derived-table alias lag_calc, and the MERGE derived-table alias clustered previously lived outside the namespace epic #137 reserves for synthesized names (giql/expander.py, EXPAND_ALIAS_PREFIX), unlike the already-prefixed __giql_cluster_id.

The genuine bug is the is_new_cluster flag — a projected column that coexists with SELECT *. A user table projecting a real column named is_new_cluster collided with the synthesized flag: the inner subquery exposed two is_new_cluster columns and the outer SUM(is_new_cluster) OVER (...) bound to the wrong one, executing silently and returning corrupted cluster ids for an integer column (or a BinderException for a non-integer one). The lag_calc / clustered renames are namespace hygiene: those are derived-table aliases in a fresh SQL scope, so they never actually shadowed a same-named user column or relation at runtime, but the reserved prefix makes the intent explicit and guards against future reuse.

This intentionally changes the emitted SQL for CLUSTER and MERGE (the synthesized alias names only), so the transpilation-string assertions and the transpilation docs' emitted-SQL example are updated in lockstep.

Closes #161

Proposed changes

Reserved-prefix rename in the CLUSTER and MERGE expanders

Rename the synthesized identifiers to __giql_is_new_cluster / __giql_lag_calc (src/giql/expanders/cluster.py) and __giql_clustered (src/giql/expanders/merge.py), along with their docstring and comment references. Each name is emitted from a single unconditional code path in its expander, so the prefix holds across every parameter variant (stranded, max-gap, predicate, nested).

Shared constants for the two producer→consumer names

Extract the two names that are aliased in one place and consumed in another — __giql_is_new_cluster (aliased in the inner CTE, consumed by the outer SUM window) and __giql_cluster_id (aliased by MERGE's CLUSTER, consumed by the GROUP BY) — to CLUSTER_FLAG_COL and CLUSTER_ID_COL in src/giql/constants.py, and consume them from cluster.py / merge.py. This mirrors how disjoin.py imports DJ_PREFIX from the same module, giving each producer→consumer pair a single source of truth so a one-sided typo cannot silently reintroduce the binder error. The single-use derived-table aliases (__giql_lag_calc, __giql_clustered) stay inline.

Transpilation-docs oracle fix

docs/transpilation/index.rst showed a half-migrated MERGE emitted-SQL example — __giql_cluster_id already prefixed alongside bare is_new_cluster / lag_calc / clustered. Update those lines to match the actual transpiler output, and quote the GROUP BY "chrom" term the transpiler actually emits.

Test cases

Update the existing transpilation-string assertions across tests/expanders/test_cluster.py, tests/expanders/test_merge.py, and tests/test_expander.py from the old bare names to the __giql_ forms, and add the coverage below.

# Test Suite Given When Then Coverage Target
1 TestClusterExpander A DuckDB table with a user column named is_new_cluster and a star-projected CLUSTER The transpiled SQL is executed It returns the correct per-partition cluster ids and preserves the user column CLUSTER collision disambiguation (executable)
2 TestMergeExpander A DuckDB table with a user column named is_new_cluster and a MERGE The transpiled SQL is executed Overlapping intervals collapse into one merged row and the separate interval into another MERGE collision disambiguation (executable)
3 TestClusterExpander A plain CLUSTER query Transpiling It aliases the synthesized helpers as __giql_lag_calc / __giql_is_new_cluster and emits no bare forms CLUSTER reserved-namespace guard
4 TestMergeExpander A plain MERGE query Transpiling It aliases the synthesized helpers as __giql_clustered / __giql_lag_calc / __giql_is_new_cluster and emits no bare forms MERGE reserved-namespace guard

Review

Round-1 review (.sdlc/reviews/issue-#161/review-1.md) returned zero blocking findings across nine reviewers; all findings were advisory. Remediated here: extracting the two producer→consumer names to shared constants (A1), softening the lag_calc / clustered comments to describe reserved-namespace hygiene rather than a runtime shadowing bug that cannot occur (A2), fixing twin comment/docstring drift (A3), quoting GROUP BY "chrom" in the docs oracle (A4), reflowing orphaned comment/docstring fragments (A5), and tightening the AAA phase boundary in the two executable tests (A8). The undocumented __giql_ namespace contract and a repo-wide reserved-identifier constant convention are deferred to follow-up #178.

https://claude.ai/code/session_01KAYsMtN7vYuECZHeeFFzCM

The CLUSTER and MERGE expanders synthesized three internal SQL
identifiers outside the reserved __giql_ namespace: the per-row
is_new_cluster flag, the CLUSTER derived-table alias lag_calc, and the
MERGE derived-table alias clustered. A user table projecting a real
column named is_new_cluster collided with the synthesized flag — the
inner subquery exposed two is_new_cluster columns and the outer
SUM(is_new_cluster) OVER (...) bound to the wrong one, executing
silently and returning corrupted cluster ids (or a binder error for a
non-integer column).

Move all three into the reserved __giql_ namespace so they cannot
collide with user-supplied names, matching the already-prefixed
__giql_cluster_id sibling. Update the emitted-SQL example in the
transpilation docs, which had drifted to a half-migrated state.

Claude-Session: https://claude.ai/code/session_01KAYsMtN7vYuECZHeeFFzCM
Update the transpilation-string assertions that pinned the old bare
lag_calc / clustered / is_new_cluster names to their reserved __giql_
forms. Add two executable DuckDB regressions — a star-projected CLUSTER
and a MERGE, each over a table carrying a user is_new_cluster column —
asserting correct cluster ids and merged rows, which proves the rename
disambiguates end-to-end (the collision executed silently and returned
wrong results before). Add a negative-leak guard per operator pinning
that no bare synthesized alias is emitted.

Claude-Session: https://claude.ai/code/session_01KAYsMtN7vYuECZHeeFFzCM
@conradbzura conradbzura force-pushed the 161-giql-prefix-cluster-merge-identifiers branch from d05ed30 to 53f9a1a Compare July 6, 2026 16:18
@conradbzura conradbzura marked this pull request as ready for review July 6, 2026 16:57
@conradbzura conradbzura merged commit 3825224 into main Jul 6, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use the reserved __giql_ prefix for CLUSTER/MERGE synthesized identifiers

1 participant