Skip to content

ci: required-check anchors for cypress-matrix and playwright-tests (unblock docs-only PRs)#40780

Merged
rusackas merged 3 commits into
masterfrom
ci/e2e-required-check-shims
Jun 5, 2026
Merged

ci: required-check anchors for cypress-matrix and playwright-tests (unblock docs-only PRs)#40780
rusackas merged 3 commits into
masterfrom
ci/e2e-required-check-shims

Conversation

@rusackas
Copy link
Copy Markdown
Member

@rusackas rusackas commented Jun 4, 2026

SUMMARY

Follow-up to #40772. Docs-only PRs currently can't merge — they hang on
cypress-matrix (0, chrome), cypress-matrix (1, chrome), and
playwright-tests (chromium) showing "Expected — Waiting for status to be
reported"
(e.g. #38108).

Root cause (same as #40772): cypress-matrix and playwright-tests are
matrix jobs gated on change detection (python || frontend). On a PR that
touches neither group — a docs-only PR — they're skipped at the job level,
which happens before matrix expansion, so the per-combination contexts are
never produced. Branch protection requires them → it waits forever.

The frontend Dependabot case (#40772) didn't expose this for the e2e jobs
because a frontend change sets frontend=true, so cypress/playwright run.
Only docs-only / neither-group PRs hit it — and the docs "build" people expect
(the Netlify Deploy Preview) does run and pass; it just isn't a required GHA
check, so the orphaned cypress/playwright contexts are what actually gate.

Fix: add always-running cypress-matrix-required / playwright-tests-required
anchors that pass when the underlying job is success or skipped, and
require those instead of the matrix-expanded names. A single
cypress-matrix-required replaces both shard contexts. The matrix jobs stay
fully skipped on unrelated PRs.

.asf.yaml:

  • cypress-matrix (0, chrome) + cypress-matrix (1, chrome)cypress-matrix-required
  • playwright-tests (chromium)playwright-tests-required

With this, every required matrix job (unit-tests, test-postgres via #40772;
cypress-matrix, playwright-tests here) has a stable anchor, so PRs of any
shape — python, frontend, or docs-only — can satisfy branch protection.

TESTING INSTRUCTIONS

  • Docs-only PR: cypress/playwright skip; the anchors run and pass → mergeable.
  • Code PR: the matrix jobs run; the anchors pass after them (and fail if any
    shard genuinely fails).

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Includes DB Migration
  • Introduces new feature or API
  • Removes existing feature or API

🤖 Generated with Claude Code

…ests

Follow-up to #40772, which added anchor jobs for the skippable matrix jobs
unit-tests and test-postgres. cypress-matrix and playwright-tests have the
same shape: they are matrix jobs gated on change detection (python ||
frontend). On a PR that touches neither — most notably a docs-only PR — the
job is skipped at the job level, before matrix expansion, so the
per-combination contexts (cypress-matrix (0, chrome), cypress-matrix
(1, chrome), playwright-tests (chromium)) are never produced. Branch
protection requires those contexts, so docs-only PRs sit forever on
"Expected — Waiting for status to be reported".

(The frontend case wasn't affected because a frontend change sets
frontend=true and these jobs run; only docs-only / neither-group PRs hit it.)

Add always-running cypress-matrix-required and playwright-tests-required
anchors that pass when the underlying job succeeded or was skipped, and point
branch protection at them (a single cypress-matrix-required replaces both
shard contexts). The matrix jobs stay fully skipped on unrelated PRs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bito-code-review
Copy link
Copy Markdown
Contributor

bito-code-review Bot commented Jun 4, 2026

Code Review Agent Run #230117

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: 980cb0b..980cb0b
    • .asf.yaml
  • Files skipped - 1
    • .github/workflows/superset-e2e.yml - Reason: Filter setting
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@dosubot dosubot Bot added the github_actions Pull requests that update GitHub Actions code label Jun 4, 2026
Comment thread .github/workflows/superset-e2e.yml Fixed
@bito-code-review
Copy link
Copy Markdown
Contributor

The security warning regarding overly broad permissions in .github/workflows/superset-e2e.yml refers to the newly added cypress-matrix-required and playwright-tests-required jobs. These jobs currently run with default permissions, which include broad access to the repository. To address this, you should explicitly define the permissions block for these jobs to follow the principle of least privilege, typically by setting permissions: contents: read.

.github/workflows/superset-e2e.yml

cypress-matrix-required:
    needs: [changes, cypress-matrix]
    if: always()
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    timeout-minutes: 5

@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.07%. Comparing base (80a3df3) to head (67c316a).
⚠️ Report is 28 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #40780   +/-   ##
=======================================
  Coverage   64.07%   64.07%           
=======================================
  Files        2664     2664           
  Lines      143786   143786           
  Branches    33072    33072           
=======================================
  Hits        92125    92125           
  Misses      50054    50054           
  Partials     1607     1607           
Flag Coverage Δ
javascript 67.59% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

zizmor flagged cypress-matrix-required and playwright-tests-required for
using default (overly broad) permissions. These jobs only read the needs
context to check a result string — no checkout, no API calls — so deny-all
`permissions: {}` is the correct least-privilege setting.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@rusackas
Copy link
Copy Markdown
Member Author

rusackas commented Jun 4, 2026

Addressed the zizmor finding in 5a6bee5 — added permissions: {} (deny-all) to both cypress-matrix-required and playwright-tests-required. They only read the needs context to inspect a result string (no checkout, no API calls), so least-privilege here is no permissions at all. Confirmed clean with zizmor .github/workflows/superset-e2e.yml locally (the only remaining warnings are pre-existing ref-version-mismatch notes on the shared actions/checkout pin, unrelated to this PR).

Comment thread .github/workflows/superset-e2e.yml
Comment thread .github/workflows/superset-e2e.yml
Per review: the anchors accepted a 'skipped' matrix result unconditionally,
but GHA also skips cypress-matrix/playwright-tests when their `needs: changes`
dependency fails or is cancelled. That would let a broken change-detector
report a false green. Require `needs.changes.result == 'success'` before
honouring a skip, so only a legitimate docs-only gate-off passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@rusackas rusackas added the merge-if-green If approved and tests are green, please go ahead and merge it for me label Jun 5, 2026
@rusackas rusackas merged commit 0984839 into master Jun 5, 2026
67 checks passed
@rusackas rusackas deleted the ci/e2e-required-check-shims branch June 5, 2026 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github_actions Pull requests that update GitHub Actions code merge-if-green If approved and tests are green, please go ahead and merge it for me size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants