Skip to content

[ci] Keep required test checks from staying expected after wiki pointer commits #230

@coisa

Description

@coisa

Problem

Pull requests can become blocked after the wiki preview workflow commits the .github/wiki submodule pointer back to the PR branch. The new head commit may contain only the wiki pointer update, so .github/workflows/tests.yml can skip its pull_request run because its paths filter does not include .github/wiki.

When that happens, branch protection still expects the required matrix checks:

  • Run Tests (8.3)
  • Run Tests (8.4)
  • Run Tests (8.5)

GitHub then leaves those checks as Expected — Waiting for status to be reported, even when the previous commit in the same PR included PHP/source changes that already needed test validation.

Current Behavior

A PR flow can look like this:

  1. A maintainer pushes a commit with PHP/docs changes.
  2. Test workflows start for that commit.
  3. The wiki preview workflow runs, publishes wiki output, and commits only the .github/wiki submodule pointer back to the PR branch.
  4. The PR head changes to the wiki pointer commit.
  5. The tests workflow is skipped by the current pull_request.paths filter because the new commit only touches .github/wiki.
  6. Required test checks for the latest head SHA remain pending/expected and the PR cannot be merged without bypassing rules.

This makes an expected workflow-managed wiki pointer update block merge instead of preserving a valid test state for the PR.

Expected Behavior

Wiki pointer commits produced by the preview workflow MUST NOT leave required test checks permanently pending.

The repository should either:

  • make required test checks report a deterministic skipped/successful status for wiki-only pointer commits; or
  • adjust the workflow/branch-protection strategy so wiki pointer commits do not become the latest SHA requiring unavailable test contexts; or
  • make the tests workflow smart enough to inspect the PR diff/base state and run when the PR contains PHP/source-sensitive changes, even if the newest commit is only .github/wiki.

Failure Surface

Affected areas:

  • .github/workflows/tests.yml, especially the pull_request.paths filter.
  • .github/workflows/wiki-preview.yml, especially the parent repository commit that updates .github/wiki.
  • Consumer workflow wrappers that inherit the same tests/wiki behavior from resources/github-actions/.
  • Branch protection rules requiring the Run Tests (8.x) matrix checks.

Proposal

Rework the skip strategy so workflow-managed wiki pointer commits cannot strand required checks in the expected state.

A likely direction is to keep a lightweight required-check workflow or job context that always reports for pull requests, then gate expensive test execution internally based on the effective PR diff instead of relying only on top-level paths filters. The implementation should also preserve the intended optimization for truly docs/wiki-only PRs.

Implementation Strategy

Inspect the interaction between:

  • the wiki preview parent commit (Update wiki submodule pointer for PR #...);
  • the tests workflow path filters;
  • required check names generated by the PHP matrix; and
  • GitHub branch protection behavior when a required workflow is skipped before creating any check run.

Then implement a deterministic strategy that ensures each required context is reported for every relevant PR head SHA. Candidate approaches include:

  • replacing top-level pull_request.paths with an always-triggered workflow plus internal path/diff detection;
  • adding a lightweight required status job that reports when expensive tests are intentionally skipped;
  • ensuring wiki pointer commits retrigger or preserve required test contexts when the PR diff includes source/test/workflow-sensitive files;
  • documenting any intentionally skipped behavior for wiki-only changes.

Non-goals

  • Do not remove wiki preview publication or the workflow-managed .github/wiki pointer update.
  • Do not relax branch protection by removing required test checks.
  • Do not force the full test matrix for PRs that truly do not require tests unless that is the smallest reliable fix.

Acceptance Criteria

Functional Criteria

  • A PR whose latest commit only updates .github/wiki no longer leaves Run Tests (8.3), Run Tests (8.4), or Run Tests (8.5) stuck as Expected — Waiting for status to be reported.
  • A PR that includes PHP/source/test/workflow-sensitive changes still runs the appropriate test validation even after the wiki preview workflow updates the pointer.
  • A PR that is truly wiki-only or docs-only keeps the intended skip behavior without blocking merge on missing required contexts.
  • Consumer workflow wrappers remain compatible with the corrected behavior.

Regression Criteria

  • Add or update workflow tests, action tests, documentation, or a reproducible manual verification note that covers the wiki-pointer commit case.
  • Verify the corrected flow on a real PR or a controlled reproduction where .github/wiki becomes the latest head change.

Architectural / Isolation Criteria

  • MUST: The core logic MUST be isolated into dedicated classes or services instead of living inside command or controller entrypoints.
  • MUST: Responsibilities MUST be separated across input resolution, domain logic, processing or transformation, and output rendering when the change is non-trivial.
  • MUST: The command or controller layer MUST act only as an orchestrator.
  • MUST: The implementation MUST avoid tight coupling between core behavior and CLI or framework-specific I/O.
  • MUST: The design MUST allow future extraction or reuse with minimal changes.
  • MUST: The solution MUST remain extensible without requiring major refactoring for adjacent use cases.
  • MUST: Data gathering or transformation MUST be isolated from filesystem writes or publishing steps.
  • MUST: Generated output ordering and formatting MUST remain deterministic across runs.
  • MUST: Re-running the workflow MUST be idempotent or clearly bounded in its side effects.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Released

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions