Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make unit test coverage decrease more significant #90

Merged
merged 5 commits into from
Jan 18, 2024
Merged

Make unit test coverage decrease more significant #90

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
9aef14e
generate-cov-report
skyshaha1 Jan 17, 2024
6a4c162
needs build-test
skyshaha1 Jan 17, 2024
ab1aaf0
commented out the upload part of build-test
skyshaha1 Jan 17, 2024
6560be9
python script - should fail as i put it to fail when unchanged
skyshaha1 Jan 17, 2024
cacb1af
removed commented stuff, changed to decrease now - should pass
skyshaha1 Jan 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 29 additions & 5 deletions .github/workflows/build_ut.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,29 @@ jobs:
name: coverage_report_stats
path: build/coverage_statistics.txt

checks-the-cov-report:
name: Checks the Coverage Report

# Runs on ubuntu
runs-on: ubuntu-22.04

# Timeout after
timeout-minutes: 2

# # Define the dependencies on the previous coverage jobs
needs: [build-test]

steps:
- name: Checkout repo
uses: actions/checkout@v3

# Download the current coverage statistics
- name: Download Current Coverage Statistics
uses: actions/download-artifact@v2
with:
name: coverage_report_stats
path: build

# Download current master coverage statistics
- name: Download Master Coverage Statistics
uses: dawidd6/action-download-artifact@v2
Expand All @@ -118,26 +141,27 @@ jobs:
branch: master
name: coverage_report_stats
path: master_artifacts

# Run the process_coverage_stats script
- name: Process Coverage Statistics
if: ${{ success() && github.ref != 'refs/heads/master' }}
run: python scripts/coverage/process_coverage_stats.py ./master_artifacts/coverage_statistics.txt build/coverage_statistics.txt

# Get process_coverage_stats script output
- id: get-comment-body
if: ${{ success() && github.ref != 'refs/heads/master' }}
if: ${{ (success() || failure()) && github.ref != 'refs/heads/master' }}
Copy link
Contributor

@skywojciechowskim skywojciechowskim Jan 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that comparison needed? It's always either success or failure

Copy link
Collaborator

@skyfroxl skyfroxl Jan 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, I think by default, test steps are skipped if status is failure or cancelled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, just tested it out on a different branch and Luke is right, test steps are skipped if the status is failure or cancelled, so both are comparisons are required

run: |
body="$(cat comparison_output.txt)"
body="${body//'%'/'%25'}"
body="${body//$'\n'/'%0A'}"
body="${body//$'\r'/'%0D'}"
echo "::set-output name=body::$body"

# Create comment with coverage info
- name: Create Coverage Comment
if: ${{ success() && github.ref != 'refs/heads/master' }}
if: ${{ (success() || failure()) && github.ref != 'refs/heads/master' }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that comparison needed?

uses: peter-evans/create-or-update-comment@v2
with:
issue-number: ${{ github.event.pull_request.number }}
body: ${{ steps.get-comment-body.outputs.body }}

8 changes: 8 additions & 0 deletions scripts/coverage/process_coverage_stats.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,14 @@ def main():
comparison_output = compare_coverage(master_stats, current_stats)
write_output(comparison_output)

# If line coverage, funtion coverage individually or both decrease, it should exit
if current_stats[0] < master_stats[0] and current_stats[1] < master_stats[1]:
sys.exit("Line coverage and function coverage have both decreased. Exiting...")
elif current_stats[0] < master_stats[0]:
sys.exit("Line coverage has decreased. Exiting...")
elif current_stats[1] < master_stats[1]:
sys.exit("Function coverage has decreased. Exiting...")

def parse_statistics(file_path):
try:
file = open(file_path, "r")
Expand Down
Loading