Skip to content

fix(gfql): preserve explicit whole-entity projection provenance - #2079

Draft
lmeyerov wants to merge 7 commits into
masterfrom
fix/gfql-entity-projection-provenance
Draft

fix(gfql): preserve explicit whole-entity projection provenance#2079
lmeyerov wants to merge 7 commits into
masterfrom
fix/gfql-entity-projection-provenance

Conversation

@lmeyerov

@lmeyerov lmeyerov commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Unlabeled whole-entity results can lack physical identity columns, leaving the adapter unable to distinguish RETURN x from property returns. This records projection kind and source presence independently of identity, preserving present entities with all-null properties, absent entities, OPTIONAL null insertion and reentry row alignment. Property/text projections clear provenance; input graphs remain unchanged.

Presence alignment uses native vectorized gather/reindex rather than a DataFrame per result segment. The paired adapter change is graphistry/tck-gfql#201; merge this product PR before that adapter PR. Tracks #2074 and graphistry/tck-gfql#200.

Final product: e1e22d196aa21b27fa748108cc49e31284252e47. Paired TCK: 9b5079eecc116dd334d32c4373903e56fc13b0a9. No scenario or expected-result snapshots changed.

Validation:

  • All 85 exact-head product CI checks succeeded. Two consecutive clean source-review waves, including pandas2/3 adversarial checks for marker/property collisions, mutation isolation, duplicate gathers, stale presence and reentry slicing.
  • Final DGX tests: cuDF25/26 each 50 passed / 14 skipped; polars-gpu 64 passed. Cases cover labels, renaming, duplicate properties with distinct IDs, DISTINCT/counts, properties, empty results and null presence.
  • Final exact-pair TCK CI: 4169 passed, 13 skipped, 689 expected failures. Local mixed pandas/native-Polars runner plus provenance tests: 3840 passed / 5 skipped / 689 expected failures; 327 native Polars declines are not execution passes.
  • Combined aggregate/validation/entity integration: 223 passed / 237 optional-backend skips. Focused pandas2/3 entity suites and supported device runs supplement this integration.

Performance: final OPTIONAL comparisons measured +6.4%/+1.58% at100/1000 seeds after replacing per-segment allocation. One separate-process ordinary-projection comparison exceeded the investigation threshold; an interleaved control measured +0.195ms/+6.12%, below the joint threshold (>20% AND >0.2ms). This retains measurement variance and modest overhead as limitations; no GPU throughput improvement is claimed.

Ruff/type-hygiene guards pass. Five local mypy dependency-profile diagnostics reproduce baseline. Exact commands, source/image pins, XML, review reports and raw timings are retained under local plans/gfql-2073-followups/. Ready for maintainer review; not merged automatically.

Current-base verification after2072

Product head10e3b4484d5f27715e10dc2a2e869e06df0101ca includes master2ca98922c; the Polars source-node-ID argument and entity presence/provenance are both preserved. Two clean catch-up source waves complete. Local entity/projection/reentry422passed; combinedTCK3840passed (327explicit Polars declines). Exact pairCI34733762397 verifies this product head with TCK9b5079eecc116dd334d32c4373903e56fc13b0a9:4169passed,13skipped,689xfail.

Earlier device/performance evidence above uses historical product heads. Product2079 is draft pending refreshed final device/performance and productCI gates. TCK201 remains dependent on2079; both remain unmerged.

@lmeyerov
lmeyerov marked this pull request as ready for review September 13, 2026 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant