Skip to content

chore: Add non-blocking test coverage advisor for PRs#13148

Merged
Cristhianzl merged 2 commits into
release-1.10.0from
cz/add-ci-tests-check
May 15, 2026
Merged

chore: Add non-blocking test coverage advisor for PRs#13148
Cristhianzl merged 2 commits into
release-1.10.0from
cz/add-ci-tests-check

Conversation

@Cristhianzl
Copy link
Copy Markdown
Member

@Cristhianzl Cristhianzl commented May 15, 2026

Objective

Warn PR authors when source code (.py/.ts/.tsx) changes ship without accompanying tests, without blocking merge or any other job.

Changes

  • Add test-coverage-advisor.yml workflow on PR open/sync/reopen/ready
  • Detect changed source vs test files using git merge-base diff (same pattern as lint-js.yml)
  • Post/update a single sticky PR comment + job summary listing files missing tests
  • Use a dedicated concurrency group and always exit 0 so it never becomes a required/blocking status

Summary by CodeRabbit

  • Chores
    • Added GitHub Actions workflow that automatically reviews pull requests and provides advisory feedback through PR comments. The workflow identifies backend and frontend source code changes, evaluates corresponding test file modifications, and offers guidance on test coverage without impacting merge operations or blocking deployments.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 15, 2026

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 45cb996f-8e55-4506-84c8-f4b700b37847

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

Adds a GitHub Actions workflow that monitors pull requests for source code changes without corresponding test coverage. The workflow detects backend Python and frontend TypeScript/TSX source file modifications, generates advisory messages indicating whether test files are present, and communicates findings via a sticky PR comment using a hidden marker.

Changes

Test Coverage Advisor Workflow

Layer / File(s) Summary
Workflow trigger and permissions
.github/workflows/test-coverage-advisor.yml
Configures the workflow to run on pull request events (opened, synchronize, reopened, ready_for_review), skips draft PRs, establishes per-PR concurrency, and grants contents read and pull_requests write permissions.
Source file change detection
.github/workflows/test-coverage-advisor.yml
Computes the PR merge base, lists all changed files, and filters backend Python sources (under src/backend and src/lfx, excluding tests/migrations/dunder) and frontend TS/TSX sources (under src/frontend/src, excluding test/spec/declaration files); outputs boolean flags and matched file lists.
Advisory message construction and PR communication
.github/workflows/test-coverage-advisor.yml
Constructs advisory-only messages indicating whether test coverage appears missing for detected source changes, mirrors output to job summary, finds or creates/updates a sticky PR comment with a hidden marker, handles fork/permission limitations gracefully, and ensures the job always succeeds.

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 9
✅ Passed checks (9 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'chore: Add non-blocking test coverage advisor for PRs' accurately and concisely summarizes the main change: adding a GitHub Actions workflow that provides advisory messages about missing tests without blocking merges.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Test Coverage For New Implementations ✅ Passed PR adds GitHub Actions workflow YAML config, not application code. Workflows tested via validation and execution, not unit tests. Custom check applies to application code, not CI/CD infrastructure.
Test Quality And Coverage ✅ Passed PR adds only GitHub Actions workflow file. Custom check targets new implementations (source code), not CI/CD configuration.
Test File Naming And Structure ✅ Passed Workflow correctly detects backend test_*.py/tests/ and frontend *.test.*/__tests__/ patterns matching repo conventions. Custom check is not applicable to CI workflow additions.
Excessive Mock Usage Warning ✅ Passed Check not applicable. The "Excessive Mock Usage Warning" check reviews test files for mock patterns, but this PR only adds a GitHub Actions workflow file and modifies no test files.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch cz/add-ci-tests-check

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Cristhianzl Cristhianzl changed the base branch from main to release-1.10.0 May 15, 2026 16:58
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 15, 2026

✅ Test Coverage Advisor

No source changes detected without accompanying tests. Thanks for keeping coverage up! 🎉

Advisory check only — never blocks merge.

@github-actions
Copy link
Copy Markdown
Contributor

Migration Validation Passed

All migrations follow the Expand-Contract pattern correctly.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 15, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.44%. Comparing base (1e6f0f9) to head (eaeb6d4).
⚠️ Report is 5 commits behind head on release-1.10.0.

Additional details and impacted files

Impacted file tree graph

@@                Coverage Diff                 @@
##           release-1.10.0   #13148      +/-   ##
==================================================
- Coverage           54.89%   54.44%   -0.45%     
==================================================
  Files                2148     2148              
  Lines              200569   200569              
  Branches            28633    30247    +1614     
==================================================
- Hits               110097   109196     -901     
- Misses              89288    90189     +901     
  Partials             1184     1184              
Flag Coverage Δ
backend 60.14% <ø> (+0.03%) ⬆️
frontend 54.32% <ø> (-0.67%) ⬇️
lfx 49.15% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 173 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Collaborator

@erichare erichare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Cristhianzl im approving, but im not sure why there's a component index update here when its just the workflow addition?

@github-actions github-actions Bot added the lgtm This PR has been approved by a maintainer label May 15, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 15, 2026

Frontend Unit Test Coverage Report

Coverage Summary

Lines Statements Branches Functions
Coverage: 38%
38.62% (48251/124915) 67.92% (6581/9688) 38.43% (1107/2880)

Unit Test Results

Tests Skipped Failures Errors Time
4321 0 💤 0 ❌ 0 🔥 8m 40s ⏱️

@Cristhianzl Cristhianzl force-pushed the cz/add-ci-tests-check branch from 668ee3e to bed901d Compare May 15, 2026 17:13
@Cristhianzl Cristhianzl enabled auto-merge May 15, 2026 17:13
@Cristhianzl Cristhianzl disabled auto-merge May 15, 2026 17:13
@Cristhianzl Cristhianzl enabled auto-merge May 15, 2026 17:14
@Cristhianzl Cristhianzl added this pull request to the merge queue May 15, 2026
Merged via the queue into release-1.10.0 with commit bd35587 May 15, 2026
115 of 116 checks passed
@Cristhianzl Cristhianzl deleted the cz/add-ci-tests-check branch May 15, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ignore-for-release lgtm This PR has been approved by a maintainer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants