Resolve genomic columns for CLUSTER/MERGE over a derived-table FROM — Closes #164

[conradbzura]

Summary

Fix genomic_columns so CLUSTER and MERGE resolve the correct physical genomic columns when their FROM clause is a derived table (or, for column resolution, a CTE), instead of silently defaulting to the canonical chrom / start / end names. For a CTE FROM this PR fixes column resolution only; end-to-end executability is not — the whole-query rewrite still drops the enclosing WITH clause, so a CTE FROM currently emits non-executable SQL. That is a pre-existing limitation tracked separately in #174.

Previously the column mapping was read only when the enclosing FROM was a bare exp.Table; a derived-table or CTE FROM yielded no table name, so the operators fell back to the canonical columns. When the underlying source used a custom mapping, the emitted SQL then partitioned/ordered/aggregated on columns that do not exist. This was pre-existing behavior carried over from the legacy ClusterTransformer and was untested (surfaced as advisory A14 in the #162 / #144 round-2 review).

The fix replaces the bare-table lookup with a recursive resolver that traces the genomic columns through a star-passthrough derived table or CTE to the underlying registered mapping. Any projection containing a star (SELECT *, SELECT alias.*, or a star alongside extra columns) passes its sources' columns through; an explicit projection is treated as transparent only when it re-exposes the canonical columns; an unregistered table or an empty FROM keeps the canonical default. When the source genuinely cannot be resolved — an opaque derived table whose columns are neither canonical nor traceable — it raises a clear ValueError rather than emitting SQL that references non-existent columns. The resolver is reached through the single genomic_columns entry point shared by both expanders, so CLUSTER and MERGE are fixed together.

Trade-offs and scope notes:

• CLUSTER/MERGE over a JOIN is unsupported and rejected with a clear ValueError. The whole-query rewrite cannot preserve a join (only the driving relation survives transplant) and bare genomic-column window/grouping references over a join are inherently ambiguous, so the shape fails loud rather than silently dropping the join. The guard fires only on the query being rewritten; a join nested inside a traced-through derived-table or CTE body is preserved and resolves from that body's driving relation.
• CTE name resolution folds identifier case (the target engines treat identifiers case-insensitively), so a mixed-case or quoted CTE reference resolves rather than silently defaulting to canonical columns.
• Resolving a CTE FROM produces the correct columns, but the whole-query rewrite still drops the enclosing WITH clause (a pre-existing limitation in transplant, orthogonal to column resolution and reproducible with a canonical table). That is deliberately left out of scope and tracked separately in CLUSTER/MERGE over a CTE FROM drops the WITH clause, emitting non-executable SQL #174; the CTE tests here assert only column resolution, not executability.

Closes #164

Proposed changes

Recursive FROM-source resolution in giql.expanders._genomic

Rewrite genomic_columns to delegate to a small resolver:

• genomic_columns rejects a top-level JOIN FROM with a clear ValueError (only the rewritten query's join would be dropped by transplant), then delegates to the resolver, raises a ValueError for an opaque/unresolvable derived source, and otherwise falls back to the canonical columns.
• _resolve_from_columns resolves the driving from_ relation to a single (mapping, opaque) result. Joins on the rewrite target are rejected upstream; a join inside a traced-through body is ignored, so its column list is taken from the driving relation.
• _resolve_source classifies a source: a CTE (looked up before registered tables and matched case-insensitively, so a CTE shadows a same-named table), a registered table, an unregistered table (assumed canonical), a derived subquery, or an untraceable source (LATERAL / VALUES / table-valued function).
• _resolve_query decides whether a derived-table/CTE body passes its columns through: any projection containing a star resolves from the inner sources; an explicit projection is transparent only when it re-exposes the canonical chrom / start / end names, otherwise the source is opaque. A set-operation body (UNION / INTERSECT / EXCEPT, via exp.SetOperation) resolves from its left arm.
• _find_cte walks the enclosing WITH clauses up the ancestor chain with a case-insensitive name match; a seen set guards against a self- or mutually-recursive CTE.
• _mapping_for reproduces the existing per-table mapping with the default-strand fallback.

Test cases

New suite tests/expanders/test_genomic_columns.py, driving the public transpile API only.

#	Test Suite	Given	When	Then	Coverage Target
1	TestGenomicColumnResolution	A CLUSTER over a SELECT * derived table on a custom-mapped table	Transpiling	Window partitions/orders/LAGs on the custom columns, no canonical names	Star-passthrough resolution (the fix)
2	TestGenomicColumnResolution	A MERGE over a SELECT * derived table on a custom-mapped table	Transpiling	Aggregation groups and MIN/MAX on the custom columns	MERGE shares the resolver
3	TestGenomicColumnResolution	A derived table projecting alias.*	Transpiling	Resolves through to the custom mapping	Qualified-star passthrough
4	TestGenomicColumnResolution	A derived table projecting a star alongside an extra column	Transpiling	Star presence still makes it a passthrough	Mixed-projection passthrough
5	TestGenomicColumnResolution	A pass-through CTE over a custom-mapped table	Transpiling	Resolves through the CTE to the custom columns	CTE column resolution
6	TestGenomicColumnResolution	A derived table wrapping another derived table	Transpiling	Recurses through both levels	Nested resolution
7	TestGenomicColumnResolution	A set-operation-bodied derived table (UNION ALL / INTERSECT / EXCEPT) with a differently-mapped right arm	Transpiling	Left arm determines the columns, no raise	Set-operation resolution
8	TestGenomicColumnResolution	A MERGE whose FROM references a CTE defined on the ancestor SELECT	Transpiling	Resolver walks up to the ancestor WITH	Ancestor-CTE lookup
9	TestGenomicColumnResolution	A CTE named identically to a registered table, over a different source	Transpiling	CTE body wins over the shadowed table	CTE-before-table precedence
10	TestGenomicColumnResolution	A source customizing only the chromosome column	Transpiling	Custom chromosome mixes with default start/end	Partial mapping
11	TestGenomicColumnResolution	A stranded CLUSTER over a derived table with a custom strand column	Transpiling	Partition includes the custom strand column	Custom strand
12	TestGenomicColumnResolution	A stranded CLUSTER over a derived table with no strand column	Transpiling	Partition falls back to the default strand column	Strand fallback
13	TestGenomicColumnResolution	A derived table re-aliasing custom columns to canonical names (CLUSTER and MERGE)	Transpiling	Resolves to canonical columns and the custom names are absent	Canonical re-expose
14	TestGenomicColumnResolution	A derived table over an unregistered source	Transpiling	Falls back to canonical columns, no raise	Unregistered default
15	TestGenomicColumnResolution	A CLUSTER with no FROM clause	Transpiling	Falls back to canonical columns	Empty-FROM default
16	TestGenomicColumnResolutionErrors	A derived table whose projection hides the genomic columns (CLUSTER and MERGE)	Transpiling	Raises ValueError explaining the source is unresolvable	Opaque-source error
17	TestGenomicColumnResolutionErrors	A VALUES-relation FROM	Transpiling	Raises ValueError	Untraceable source
18	TestGenomicColumnResolutionErrors	A table-valued-function (UNNEST) FROM	Transpiling	Raises ValueError	Untraceable source
19	TestGenomicColumnResolutionErrors	A mutually-recursive CTE	Transpiling	Cycle guard raises rather than looping	Recursion guard
20	TestGenomicColumnResolutionErrors	A top-level JOIN of two relations	Transpiling	Raises ValueError that a join is unsupported	Join rejection
21	TestGenomicColumnResolutionErrors	A top-level JOIN mixing a concrete-mapped and an opaque source	Transpiling	Raises ValueError that a join is unsupported	Join rejection
22	TestGenomicColumnResolutionErrors	A top-level JOIN of tables with conflicting mappings (CLUSTER and MERGE)	Transpiling	Raises ValueError that a join is unsupported	Join rejection
23	TestGenomicColumnResolutionErrors	A qualified alias.* projected over a top-level JOIN	Transpiling	Raises ValueError that a join is unsupported	Join rejection

https://claude.ai/code/session_01KAYsMtN7vYuECZHeeFFzCM