Skip to content

test(gfql): stop test_index.py fabricating 20 failures without cudf_polars - #1811

Merged
lmeyerov merged 3 commits into
masterfrom
audit/polars-lane-orphans-ca4
Jul 28, 2026
Merged

test(gfql): stop test_index.py fabricating 20 failures without cudf_polars#1811
lmeyerov merged 3 commits into
masterfrom
audit/polars-lane-orphans-ca4

Conversation

@lmeyerov

@lmeyerov lmeyerov commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Rescoped after discovering an overlap — please read this first.
This branch originally carried the whole CA4 allowlist change (seven files added to bin/test-polars.sh). #1805 (fix/gfql-polars-lane-invisible-tests), opened concurrently, is a superset of that list and adds test_polars_lane_completeness.py — a guard that fails when a polars-parametrized file is missing from the lane, which makes the fix permanent rather than one-time. It also lands the same pandas-3 case-mapping xfail against the same issue (#1802). Take #1805 for CA4. This PR is reduced to the one hunk #1805 does not contain, so the allowlist is reviewed once instead of twice.

The remaining change

graphistry/tests/compute/gfql/index/test_index.py built its engine list by appending "polars-gpu" when cudf imported:

try:
    import cudf  # noqa
    out.append("polars-gpu")
except Exception:
    pass

But polars-gpu is the cudf_polars GPU collect target, and without it GFQL raises:

ImportError: GFQL engine='polars-gpu' requires the RAPIDS cudf_polars stack (NVIDIA GPU).

So any box with cuDF installed and cudf_polars absent — a common developer configuration, and mine — gets 20 failures indistinguishable from product breakage, in a file that is listed in bin/test-polars.sh.

CI never caught it because that lane installs neither package, so the polars-gpu parameter did not exist there. A gate can only fabricate failures where it is never exercised — which is the same blind spot as #1795, one level down: not "a file that runs nowhere", but "a parameter that exists only where nobody looks".

Measured

Box with cuDF 25.10, no cudf_polars:

before:  20 failed, 116 passed
after:    0 failed, 105 passed

The 15-test difference is the polars-gpu parameters correctly no longer being generated; they still generate — and pass — on a box with the full RAPIDS stack.

Runtime delta: zero

One test file's engine list and a CHANGELOG line. Per CB5 no pyg-bench lane run is required, and that is checkable from the diff.

What the (now-withdrawn) allowlist work found, for the record

Running the assembled lane locally and on GPU before rescoping produced evidence worth keeping, since #1805 will inherit these files:

  • Full lane, cuDF absent (the CI polars lane's own configuration): 2658 passed, 58 skipped, 0 failed in 160s, plus the appended cypher-lowering pass at 49 passed. The lane's timeout is 10 min and it runs twice (plain + coverage); the added files cost ~15s per run.
  • dgx GB10, graphistry/test-rapids-official:26.02-gfql-polars, docker run --gpus all, cuDF 26.02.01 / polars 1.35.2, every cuDF and polars-gpu parameter live: 394 passed, 0 failed, 0 skipped.
  • test_indexed_bindings.py::test_explicit_polars_gpu_declines_indexed_helper_and_falls_back has no GPU guard — building polars frames does not need the RAPIDS stack, so the graph constructs fine and the run dies later inside .gfql(). test(gfql): make the polars lane see the 280 tests it was skipping everywhere #1805 fixes this in the same file; flagging it so it is not lost.
  • The 37 failures these files show on a box with cudf importable but libnvrtc.so.12 missing are that environment, not the product — they reproduce nowhere with a working CUDA runtime. Re-running on real hardware before reporting is why the conclusion is "no product failures" rather than seven issues.

🤖 Generated with Claude Code

https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB

@lmeyerov

Copy link
Copy Markdown
Contributor Author

Update: the first CI run found what this PR exists to find

test-polars went red on exactly two cases, and they are a real cross-engine divergence, not a fixture problem:

2 failed, 2657 passed, 103 skipped
  test_viz_pipeline_conformance.py::test_case_regex_unicode_trick_matrix[toupper-eq-ss-fold]
  test_viz_pipeline_conformance.py::test_case_regex_unicode_trick_matrix[tolower-turkish-dotted-i]

The polars lane resolves pandas 3.0.3; local dev and test-gfql-core resolve 2.3.3. pandas 3 defaults to a PyArrow-backed str dtype whose utf8_upper/utf8_lower do simple Unicode case mapping, where Python, polars and pandas 2 do full mapping:

input Python polars 1.42 pandas 2.3.3 pandas 3.0.3
'ß'.upper() 'SS' 'SS' 'SS' 'ẞ' (U+1E9E)
'İ'.lower() 'i̇' 'i̇' 'i̇' 'i'

So MATCH (n) WHERE toUpper(n.name) = 'STRASSE' returns a different row set on pandas 3 than on polars, with no error — and pandas is the oracle the parity suites compare against.

Filed as #1812. Not fixed here: it needs an owner call on which mapping is correct (Neo4j uses Java's full mapping; openCypher does not pin the algorithm; matching it on pandas 3 costs a .map(str.upper) on the hot path).

Pinned with xfail(strict=True) conditioned on a runtime probe of the backend, applied per-case — everything else in that matrix is casing-agnostic and must keep passing on both. On pandas 2 the condition is False and the marker is inert, so the cases must pass; on pandas 3 they must fail. Whoever reconciles the backends is told to delete the marker rather than finding a stale skip.

Worth stating plainly, because it is the argument for this PR: that divergence has been in the tree with a test already written for it, and the test never ran.

@lmeyerov
lmeyerov force-pushed the audit/polars-lane-orphans-ca4 branch from 458498f to 66a462f Compare July 28, 2026 02:08
@lmeyerov lmeyerov changed the title test(gfql): seven test files that ran in NO CI lane at all now run (CA4 / #1795) test(gfql): stop test_index.py fabricating 20 failures without cudf_polars Jul 28, 2026
…olars

`_engines()` appended `"polars-gpu"` when **`cudf`** imported. But `polars-gpu`
is the `cudf_polars` GPU collect target and raises

    ImportError: GFQL engine='polars-gpu' requires the RAPIDS cudf_polars stack

without it. So any box with cuDF installed and `cudf_polars` absent — a common
developer configuration — got 20 failures indistinguishable from product
breakage, in a file that IS listed in `bin/test-polars.sh`. CI never caught it
because that lane installs neither package, so the parameter did not exist
there: a gate can only fabricate failures where it is never exercised.

Measured on a box with cuDF 25.10 and no `cudf_polars`:
  before  20 failed, 116 passed
  after    0 failed, 105 passed

SCOPE NOTE. This branch originally carried the whole CA4 allowlist change
(seven files added to `bin/test-polars.sh`). PR #1805
(`fix/gfql-polars-lane-invisible-tests`), opened concurrently, is a SUPERSET of
that list AND adds `test_polars_lane_completeness.py`, a guard that fails when a
polars-parametrized file is missing from the lane — which makes the fix
permanent rather than one-time. It also lands the same pandas-3 case-mapping
xfail, against the same issue. Reduced to the one hunk #1805 does not contain,
so the owner reviews the allowlist once instead of twice.

RUNTIME DELTA: zero. One test file's engine list and a CHANGELOG line; no
pyg-bench lane run required (CB5), checkable from the diff.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB
@lmeyerov
lmeyerov force-pushed the audit/polars-lane-orphans-ca4 branch from 66a462f to cee1dd0 Compare July 28, 2026 03:01
lmeyerov added 2 commits July 28, 2026 02:32
Resolves the CHANGELOG.md conflict: master added an Infrastructure
section and new Changed/Fixed/Documentation entries under
[Development] while this branch added one Changed entry. Took master's
file and re-inserted this branch's entry at the head of ### Changed.

graphistry/tests/compute/gfql/index/test_index.py is untouched on
master, so the branch side carries through unchanged.

Verified against master: test_index.py IS listed (exactly once) in
bin/test-polars.sh POLARS_TEST_FILES, and the test-polars lane installs
the `test,polars` extras only, so the CHANGELOG claims still hold.
Resolves the CHANGELOG.md conflict only. Master added an Infrastructure
section plus new Changed/Fixed/Documentation entries under
[Development]; this branch adds one ### Changed entry. Took master's
file verbatim and re-inserted this branch's entry at the head of
### Changed.

graphistry/tests/compute/gfql/index/test_index.py is untouched on
master, so the branch side carries through unchanged. This branch does
not modify bin/test-polars.sh.

Verified against this master: test_index.py is listed exactly once in
bin/test-polars.sh POLARS_TEST_FILES (so the CHANGELOG's "a file that
IS in the polars lane" claim holds), and the test-polars lane installs
the `test,polars` extras only on ubuntu-latest, so "that lane installs
neither package" holds too.
@lmeyerov
lmeyerov merged commit b6181d3 into master Jul 28, 2026
57 checks passed
lmeyerov added a commit that referenced this pull request Jul 28, 2026
Only CHANGELOG.md changed on both sides. Resolved structurally, not
textually: took master#\s file whole and re-inserted this branch#\s single
### Changed entry at the top of that section, so the diff vs master is
+1/-0 on CHANGELOG. The eight source files this branch retypes were
untouched by #1811 (which only moved test_index.py and the CHANGELOG),
so they carry through unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YYZRXegrALuXd3NHH5evqx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant