CI: path filters over-trigger — any .py change runs the full GFQL/pandas/networkx matrix

[lmeyerov]

Symptom

A one-line change to a single unrelated plugin test (graphistry/tests/plugins/test_networkx.py, PR #1686) triggered test-gfql-core (×2 py versions), test-pandas-compat* (×5), the full test-networkx-scipy-policy matrix (×3), lint/types (×7), and more — dozens of jobs / many CI-minutes for a change that touches nothing those suites exercise.

This isn't specific to that PR; it happens for essentially every code PR. Worth a look at the changes path-filter design in .github/workflows/ci.yml.

Mechanism

The changes job computes boolean path filters, and heavy jobs gate on them. For that networkx-test change:

filter	value	why
python	true	its patterns are \.py$ and ^graphistry/ — either alone matches almost any code file
gfql	false	^graphistry/compute/gfql/…; a plugin test doesn't match
cypher_frontend_ci	false	—

But test-gfql-core (and the networkx/pandas matrices) gate on:

if: needs.changes.outputs.python == 'true'
 || needs.changes.outputs.gfql == 'true'
 || needs.changes.outputs.ai == 'true'
 || needs.changes.outputs.infra == 'true' || …

So python alone is enough to run them.

Three systemic levels

1. python is a catch-all, not a discriminator. \.py$ matches every Python file in the repo, so python=true for practically every code PR. It carries no signal beyond "some code changed."
2. Filters are additive-only, never subtractive. The narrow lanes (gfql, cypher_frontend_ci, ai, spark) can only add jobs on top of what python already forces. Because heavy jobs OR in python, a false gfql/cypher_frontend_ci never removes anything. The narrow filters only bite for the rare pure-non-.py change (docs, a lock file).
3. No lane for isolated/plugin tests. A change confined to graphistry/tests/plugins/* has no bucket that runs only the relevant plugin job — it falls into the coarse python catch-all.

Direction to consider (tradeoff is yours)

Make heavy lane jobs gate on their narrow filter plus a small shared-core filter, instead of the blanket python:

• Add a core filter for genuinely cross-cutting files: ^graphistry/Engine\.py$, ^graphistry/PlotterBase\.py$, ^graphistry/Plottable\.py$, ^setup\.py$, ^\.github/workflows/ci\.yml$, shared compute/ infra, lockfiles.
• test-gfql-core → gfql || core || infra (drop bare python).
• test-networkx-scipy-policy → a networkx/plugins narrow filter || core (drop bare python).
• Optionally a plugins lane (^graphistry/(plugins|plugins_types)/, ^graphistry/tests/plugins/) scoped to just the plugin jobs.

Net effect: a networkx-only change runs just the networkx job; a GFQL change runs GFQL; a shared-core change still fans out (correctly).

Correctness caveat (the real work): the current conservative python gate is a deliberate "when unsure, run it" choice. Tightening it risks skipping a suite that a shared-code change actually needs (e.g. a change to a util under graphistry/ that GFQL imports but that isn't under compute/gfql/). The valuable part of this issue is auditing which shared paths each lane truly depends on and encoding that as the core filter — not the mechanical gate edit. Suggest starting from a couple of real recent PRs and checking "what ran vs what needed to run."

Note: ci.yml itself is in the infra filter, so any change here re-triggers the full matrix (expected for a CI edit).

Evidence

• PR fix(test): networkx HITS test used a degenerate cycle graph (scipy-version flaky) #1686 (the trigger): fix(test): networkx HITS test used a degenerate cycle graph (scipy-version flaky) #1686
• Surfaced while landing the GFQL Cypher LALR parser PR (feat(gfql/cypher): Earley → LALR(1) parser (~70× faster parse) with machine-checked grammar invariants #1682), whose CI was red only on the unrelated nx-upper-scipy job.

🤖 filed via Claude Code