ci: speed up job matrix (issue #1082)#1084
Merged
Conversation
…nsolidation, core-py profiling)
…fs/dgl steps - test-minimal-python sentinel: switch to bin/test-minimal-lite.sh (excludes test_hyper_dask, test_compute_chain, test_chain_let, test_hop, test_plotter). Target: <60s gate (was 231s). Deferred files run in test-minimal-python-rest. - test-minimal-python-rest: now runs full bin/test-minimal.sh including deferred heavy files, covering all 7 versions in parallel with downstream. - test-full-ai: consolidate 4 sequential shell scripts into single pytest -n auto call. Eliminates 3x pytest startup + redundant HF model warm-up overhead. - test-dgl-cpu: remove duplicate direct pytest call (test-embed.sh/test-dgl.sh already run the same files). - All jobs: remove 9 redundant 'git lfs pull' steps (checkout lfs:true suffices). - timeout-minutes on sentinel: 15 → 5 (matches new target). Closes part of #1082. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…, should be ~1700) bin/test-minimal.sh was missing --ignore for the 3 GFQL-core files that have their own dedicated job (test-gfql-core). pandas-compat calls test-minimal.sh directly so was running ~1400 extra GFQL tests = ~200s wasted per job. Fix: bake GFQL ignores into test-minimal.sh and test-minimal-lite.sh so any caller gets the right scope. Simplify CI step to drop redundant --ignore args. Expected: pandas-compat 429s → ~200s. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New parallel job runs GFQL core tests against pandas 2.2.3 (py3.9) and pandas>=3.x (py3.14). Skips rapids-aligned (2.3.3/py3.13) since test-gfql-core already resolves ~2.3.x via uv for those versions. Adds test-compat-gfql-legacy and test-compat-gfql-latest lockfile profiles to generate-lockfiles.sh. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test-pandas-compat: replace full test-minimal.sh run (~1700 tests) with explicit pandas-API-sensitive files only (~400-600 tests): test_plotter, test_compute_chain, test_hypergraph, test_compute_hops, test_compute_collapse, test_compute_filter_by_dict, test_bolt_util, test_nodexl. ~65% reduction in test count per matrix row. test-pandas-compat-gfql: expand from 3 query-layer files to full GFQL stack: query layer + execution engine (test_hop, test_chain, test_chain_let, test_chain_concat, test_dataframe_primitives) + schema layer (test_schema_changers, test_let_schema_changers). Fixes gap where test_chain_let (62 pandas hits) had no compat coverage. test-gfql-core: same expansion — now covers full GFQL stack at baseline pandas (~2.3.x resolved by uv). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This was referenced Apr 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1082.
What changed and why
Sentinel gate split
test-minimal-python(231s, critical path gate) was split into:test-minimal-lite.sh— fast smoke gate excluding 5 heavy files + GFQL-core files. Gates all downstream jobs.test-minimal-python-restpool (runs in parallel with downstream).Critical path: 719s → ~550s (−23%).
test-pandas-compat redesigned (coverage + speed)
Before: ran
bin/test-minimal.sh— ~1700 tests per matrix row, most with no pandas API sensitivity. Wasted CI minutes running pandas-insensitive tests 3× across the version matrix.After: explicit pandas-API-sensitive file list only (~400–600 tests per row):
test_plotter.py— core DataFrame ingestion, binding, type coerciontest_compute_chain.py— chain computation (105 pandas API hits)test_hypergraph.py— hypergraph constructiontest_compute_hops.py,test_compute_collapse.py,test_compute_filter_by_dict.py— compute layertest_bolt_util.py,test_nodexl.py— connectors/formatsMatrix: py3.9+pandas 2.2.3 (legacy), py3.13+pandas 2.3.3 (rapids-aligned), py3.14+pandas 3.x (latest).
~65% fewer tests per row. Pandas-insensitive tests are covered by
test-core-pythoninstead.test-pandas-compat-gfql (new job)
Full GFQL stack tested across pandas edge versions (2.2.3/py3.9 and 3.x/py3.14). Skips rapids-aligned since
test-gfql-corealready resolves ~2.3.x via uv.Covers:
gfql/cypher/test_lowering.py,test_parser.py,gfql/test_row_pipeline_ops.pycompute/test_hop.py,test_chain.py,test_chain_let.py,test_chain_concat.py,test_dataframe_primitives.pygfql/test_schema_changers.py,gfql/test_let_schema_changers.pyFixes coverage gap:
test_chain_let.py(62 pandas hits) had no pandas-version coverage before.test-gfql-core expanded
Previously only ran 3 query-layer files. Now runs the full GFQL stack (same file list as
test-pandas-compat-gfql) at baseline pandas (~2.3.x resolved by uv) on py3.12 + py3.14.test-full-ai consolidated
4 sequential shell scripts → single
pytest -vv -n autoinvocation. Eliminates 3× pytest startup + HuggingFace model cold-start overhead.Expected: ~300s → ~150s.
Other cleanups
git lfs pullsteps (checkout already haslfs: true)test-dgl-cpuduplicate test runs (was running same files via direct pytest then again via shell scripts)PIP_EXCLUDE_NEWER: "6 days"globally (alongside existingUV_EXCLUDE_NEWER)Before / After
Confirmed green run: https://github.com/graphistry/pygraphistry/actions/runs/24059417800
🤖 Generated with Claude Code