Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
- **Seeded typed-hop property projection keeps its fast path through DISTINCT / ORDER BY / SKIP / LIMIT**: those trailing row ops are plain frame operations, so the fast path now delegates them to the canonical chain instead of declining the whole query.

### Changed
- **The polars-engine index tests stop fabricating 20 failures on a cuDF-without-`cudf_polars` box** (tests only; zero runtime delta). `graphistry/tests/compute/gfql/index/test_index.py` built its engine list by appending `"polars-gpu"` when **`cudf`** imported — but `polars-gpu` is the `cudf_polars` GPU collect target and raises `ImportError: GFQL engine='polars-gpu' requires the RAPIDS cudf_polars stack` without it. So any environment with cuDF installed and `cudf_polars` absent (a common developer configuration) got 20 failures indistinguishable from product breakage, in a file that IS in the polars lane. CI never caught it because that lane installs neither package, so the parameter did not exist there — a gate can only fabricate failures where it is never exercised. The gate now tests `cudf_polars`. Measured: 20 failed / 116 passed before, 105 passed / 0 failed after, on a box with cuDF 25.10 and no `cudf_polars`.
- **`rows(table=...)` and rewrite-param regression suites now run on all four engines** (tests only; zero runtime delta — no production line changed). The #1788 and #1790 suites were parametrized over pandas + polars only, so the `table` guards they exist to pin were never exercised on cuDF or polars-gpu at all. Both files were already in `bin/test-polars.sh` (#1805), so their polars params did run; what ran nowhere was `test_rows_table_named_middle.py` outside that one lane — a module-level `pytest.importorskip("polars")` skipped the whole file in `test-gfql-core`, which installs no polars. Both files are now parametrized over pandas / cuDF / polars / polars-gpu from a **fixed** engine list plus a classified availability check; dropping the module-level `importorskip` also makes 5 pandas cases live in `test-gfql-core`. The availability check is deliberately not `available_nonpandas_engines()`, which builds its list by importability so a missing engine vanishes from the report instead of showing as SKIPPED, and it is deliberately not a blanket `except Exception` either: a missing module skips, a recognisable GPU-stack error skips **with its text quoted**, and any other failure propagates — because a skipped GPU parameter reads as evidence of passing. Both traps were hit by trying rather than reasoned about: a probe that ran the shape under test turned a reverted production guard into 20 SKIPS instead of 20 failures, and a blanket probe silently dropped cuDF from an otherwise-green GPU run on a transient `cudaErrorMemoryAllocation`. GPU receipts on dgx GB10 / cuDF 26.02.01 / polars 1.35.2 / `docker run --gpus all`: **45 passed, 3 xfailed, 0 skipped**; mutation-checked by reverting the #1788 and #1790 `table` guards — **24 failures spread evenly over all four engines (6 each)**, so the added parameters are not decorative. Two gaps the new parameters surfaced are pinned as strict xfails rather than papered over: #1803 (the indexed bindings bypass excludes `Engine.POLARS_GPU`, so polars-gpu silently takes the scan path) and #1804 (the native polars bindings builder never receives `alias_prefilters` — previously an unnumbered xfail).
- **Row execution context: `clear_row_exec_context` NULLs rather than restores, and that is now pinned rather than assumed** (tests + comments only; zero runtime delta — no production line changed). Review of #1793 asked whether the exit should RESTORE what the graph carried on entry instead of nulling, since `attach_row_exec_context` INHERITS on the way in (a `None` argument keeps what `g` already carries). Settled as NULL, for three reasons now each carried by a test: (1) `clear` is pure — it returns `g.bind()` and never writes through to the object it was handed, so an outer scope that set the field still has it afterwards and there is no caller state to save; that is what separates this from #1786, which WAS an in-place write onto the caller's own graph, and restore is the fix for a mutation. (2) The only channel restore would change is the RETURN value, and putting the seed back there IS the second half of #1786 — measured, a seed hand-restored onto a result changes the answer of the next query run on that result (7 → 2 → 1 rows). (3) No execution frame inherits a context it did not set: instrumenting `attach` over `graphistry/tests/compute` recorded 3907 calls and zero inheriting ones (53 entered on a graph already carrying a seed, but each was handed the identical `start_nodes` parameter, so the frame still owns the value), because the cross-segment `WITH` seed travels as the explicit `start_nodes` PARAMETER and never through the graph field. The new `test_exec_context_scoping.py` re-runs that ownership measurement as an assertion, so a future path that starts relying on inheritance reopens the decision loudly instead of silently losing an outer value. Mutation-checked in the deciding direction: implementing save/restore at all three attach sites fails 4 of the new tests on every runnable engine, while the existing #1793 suite passes unchanged — i.e. those tests could not distinguish the two designs and this file can. Engine-parametrized over pandas/cuDF/polars/polars-gpu from a fixed list plus a classified availability check (a missing module skips, a recognisable GPU-stack error skips with its text quoted, and any other failure propagates — a silently skipped GPU parameter reads as evidence of passing); GPU receipts on dgx GB10 / cuDF 26.02.01 / `docker run --gpus all`: 43 passed, 0 skipped.
- **`is_lazy` is a type predicate, so the polars engine's eager/lazy split is checked instead of asserted**: `dtypes.is_lazy` decides WHICH member of the two-member `PolarsFrame` union a frame is, but it was declared `-> bool`, so that answer was discarded at the call boundary — an eager-only attribute (`.height`, `.columns`, `.schema`) reached after a lazy guard was unprovable, and the native polars chain closed the gap with `cast("pl.DataFrame", frame)`. A cast is not a type: it re-asserts, unchecked, the exact fact the predicate had just established, once per call site, and a wrong one surfaces as a production `AttributeError` rather than a CI error. `is_lazy` is now declared `-> TypeIs["pl.LazyFrame"]` (PEP 742), which narrows the *negative* branch as well as the positive — the eager side is the one that needed it, so `TypeGuard` would not have done — and the cast is gone. Four previously untyped chain-combine helpers (`_semi`, `_combine_edges`, `_apply_node_names`, and the `_known_empty` guard) now carry real signatures: `_semi`'s two frames share the `PolarsT` TypeVar because polars joins do not mix eagerness, and the two combine helpers take the `_LazyShim` collect-once duck-type they are actually called with. Typing `_apply_node_names` also retired a `getattr(next_step, "edges_empty", None)` — the shim declares that tri-state in `__slots__`, so a typo is now a checker error instead of a silent `None` that would have re-armed the very cardinality gate the guard disarms. Internal only: no runtime behaviour, no public API, and the `TypeIs` import is `TYPE_CHECKING`-only, so no new runtime dependency floor.
Expand Down
8 changes: 7 additions & 1 deletion graphistry/tests/compute/gfql/index/test_index.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,13 @@ def _engines():
import polars # noqa
out.append("polars")
try:
import cudf # noqa
# `cudf_polars`, NOT `cudf`: engine='polars-gpu' is the cudf_polars GPU collect
# target and raises `ImportError: GFQL engine='polars-gpu' requires the RAPIDS
# cudf_polars stack` without it. Gating on `cudf` FABRICATED 20 failures on every
# box that has cuDF but not cudf_polars — a real configuration, and the failures
# are indistinguishable from product breakage. CI never caught it because its
# polars lane installs neither, so the parameter simply did not exist there.
import cudf_polars # noqa
out.append("polars-gpu")
except Exception:
pass
Expand Down
Loading