Cypher/GFQL: replace bounded reentry hidden-column handshake with an explicit ReentryPlan

[lmeyerov]

Problem

Bounded MATCH ... WITH ... MATCH ... reentry currently works, but the internal design is hard to reason about.

Today the same concept is spread across multiple mechanisms:

• start_nodes_query in graphistry/compute/gfql/cypher/lowering.py
• hidden __cypher_reentry_* columns and expression rewrites in graphistry/compute/gfql/cypher/lowering.py
• _cypher_entity_projection_meta side-channel metadata
• _compiled_query_reentry_state() stitching logic in graphistry/compute/gfql_unified.py

That makes the compiler/runtime contract implicit instead of explicit. A senior compiler / graph language / GPU engineer joining the project would have to reconstruct the model from several places at once.

Why This Matters

• harder to audit vectorization and backend purity
• harder to extend to the next row-seeded features
• hidden invariants across compiler + runtime increase maintenance risk
• lowering.py and gfql_unified.py are longer and conceptually denser than they need to be

Proposed Refactor

Treat bounded reentry as a first-class plan/runtime concept rather than a protocol assembled from side channels.

Recommended steps:

1. Introduce an explicit ReentryPlan (or SeededMatchPlan) dataclass.

   • carried alias
   • id column
   • carried scalar outputs
   • ordering contract
   • trailing match alias contract

2. Replace the current hidden-property rewrite protocol.

   • stop encoding carried scalars as synthetic __cypher_reentry_* property accesses
   • instead carry an explicit scalar mapping in the plan contract

3. Move runtime stitching into a dedicated reentry module.

   • keep gfql_unified.py as dispatch/orchestration
   • move reentry-specific assembly/validation into a smaller targeted runtime helper module

4. Make row-order and seed-row semantics explicit.

   • preserve order as part of the contract, not as an inferred merge behavior

5. Split lowering.py by concern where useful.

   • general lowering
   • result projection planning
   • bounded reentry planning

Non-Goals

• no semantic expansion required in this issue
• keep this separate from PR fix: extend bounded cypher reentry carrier #975 unless review says the current shape is not landable
• do not broaden generic WITH semantics here

Success Criteria

• existing bounded-reentry semantics stay green
• current pandas + cudf bounded-reentry tests stay green
• the reentry contract becomes readable from one place
• low-hundreds LOC reduction across lowering.py + gfql_unified.py is plausible from collapsing duplicate protocol layers
• follow-on work for multi-alias row carriers / optional null-extension becomes easier to reason about

Context

Current bounded-reentry hardening/validation work is in PR #975.
This issue is the follow-on cleanup/refactor lane, not a request to reopen that PR scope.

[lmeyerov]

Related broader architecture track: #989 covers the general row-carrier / seeded-row IR direction beyond the bounded-reentry-specific cleanup in this issue.

[lmeyerov]

Step 1 landed — ReentryPlan dataclass introduced

PR #1248 squashed to master as 80d80849c introduces the ReentryPlan + CarriedAlias dataclass at graphistry/compute/gfql/cypher/reentry_plan.py, exposed via compiled_query.reentry_plan and threaded through _map_terminal_reentry_query + _attach_graph_context.

The plan is now the source of truth at the compile/runtime boundary for whole-row aliases (one is_reentry_alias=True, others as carried), top-level scalar carries, and the scalar-only prefix shape.

Status of the steps in this issue

• Step 1 — introduce ReentryPlan: ✅ landed (feat(cypher): ReentryPlan + multi-whole-row prefix WITH (#989 slices 4.1+4.3a) #1248).
• Step 2 — replace hidden-column-handshake protocol with explicit plan field: 🟡 partially started. New code paths read from ReentryPlan, but the legacy scalar_reentry_alias / scalar_reentry_columns fields on CompiledCypherExecutionExtras and the tuple return from _bounded_reentry_carry_columns still co-exist. Removing them is queued for the next slice once Cypher/GFQL rejects multi-stage MATCH ... WITH ... MATCH ... WITH ... MATCH read queries #999 IC3 work (slice 4.3d) lands and the runtime fully consults the plan.
• Step 3 — move bounded-reentry runtime stitching out of gfql_unified.py: ❌ not started. Pure-move refactor; deferred until plan adoption is complete to avoid moving code twice.
• Steps 4–5 — scalar-field removal, cleanup: ❌ not started.

Issue stays open; next slice on this lane will follow once slice 4.3d (cross-reentry-boundary carry forwarding for #999 IC3) lands and exercises the plan end-to-end.