Skip to content

Closes #739: Skip slow and external network tests in PR CI - #740

Merged
luisremis merged 2 commits into
developfrom
fix/739-reduce-ci-time
May 24, 2026
Merged

Closes #739: Skip slow and external network tests in PR CI#740
luisremis merged 2 commits into
developfrom
fix/739-reduce-ci-time

Conversation

@ad-claw000

Copy link
Copy Markdown
Contributor

Closes #739

This PR addresses the slow CI execution times in the PR workflow by implementing Option 3 from the issue: Separating 'Slow' / 'External' Tests from PR workflows.

Why Option 3 over the others?

  1. Option 1 (Sharding pytest): Pytest sharding using pytest-split or pytest-xdist would reduce wall-clock time, but it introduces the complexity of aggregating coverage across jobs. It also runs the risk of race conditions if concurrent test processes mutate the same ApertureDB database state.
  2. Option 2 (Caching): We already utilize DOCKER_BUILDKIT: 1 and layer caching via GitHub Container Registry (see pr.yaml). While we could attempt to cache Pip more granularly, the actual Pytest execution phase remains the largest bottleneck.
  3. Option 3 (Separating slow tests): By simply setting an environment variable (SKIP_SLOW_TESTS=true) exclusively in the PR workflow, we gain immediate and significant reduction in CI completion times. The tests marked with @pytest.mark.slow or @pytest.mark.external_network will be skipped in PR feedback loops, but will continue running in the daily dependencies.yml runs and develop.yml / main.yml workflows to ensure no coverage is lost globally.

Implementation details

  • Modified pr.yaml to pass SKIP_SLOW_TESTS=true
  • Updated run_test_container.sh to forward this environment variable directly to the docker run commands executing tests
  • Updated run_test.sh to inject and not slow and not external_network into the pytest $FILTER query when the flag is true

This reduces the feedback loop duration for pull requests by skipping the slower and external_network tests from the pr.yaml workflow.
Copilot AI review requested due to automatic review settings May 23, 2026 21:33
@ad-claw000 ad-claw000 self-assigned this May 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces pull-request CI runtime by skipping tests marked as slow and external_network only in the PR workflow, while keeping full coverage in other workflows (e.g., scheduled or branch workflows).

Changes:

  • Adds SKIP_SLOW_TESTS=true to the PR GitHub Actions workflow environment.
  • Forwards SKIP_SLOW_TESTS into the test Docker containers.
  • Updates the test runner to extend the pytest marker expression to exclude slow and external_network when the flag is enabled.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
test/run_test.sh Conditionally modifies the pytest -m marker filter to exclude slow and external_network when SKIP_SLOW_TESTS is enabled.
test/run_test_container.sh Passes SKIP_SLOW_TESTS through to the Docker containers running the tests.
.github/workflows/pr.yaml Sets SKIP_SLOW_TESTS: true for PR CI runs to speed up feedback loops.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@luisremis luisremis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you need to add the nightly workflow, which runs all the tests, as part of this pr.

@ad-claw000

Copy link
Copy Markdown
Contributor Author

Added nightly.yml workflow to run all tests on a schedule, as requested. See commit 88f1e44.

@luisremis
luisremis merged commit 12cbe4d into develop May 24, 2026
3 checks passed
@luisremis
luisremis deleted the fix/739-reduce-ci-time branch May 24, 2026 05:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate and reduce CI execution times for PR workflow

3 participants