Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Add CodeCov.io flags to disambiguate reports #5788

Merged
merged 1 commit into from Mar 29, 2021

Conversation

electronjoe
Copy link
Member

@electronjoe electronjoe commented Mar 29, 2021

Summary

We are observing erratic coverage reporting as CodeCov.io wants every git hash being reported to have complete coverage statistics - which we are not doing for latency / CPU / cost reasons (e.g. if a PR does not touch c/c++ targets we are omtting the hour+ of testing and upload of coverage). This PR attempts to use per-coverage-domain codecov.io flags to alleviate the issue (why do they call them flags and not tags! ARG).

The strategy is to apply a codecov.io project flag per CircleCI pipeline stage that pushes a coverage report - to ensure each can be run in isolation of any others.

Test Plan

Test-ability of CircleCI / CodeCov configuration does not seem great - particular for the multi-target PR such as this. I think given the current broken codecov behavior, we might as well merge this and then iterate.

Signed-off-by: Scott Harrison Moeller smoeller@fb.com

@electronjoe electronjoe requested a review from tmdzk as a code owner March 29, 2021 14:59
@pull-request-size pull-request-size bot added the size/M Denotes a PR that changes 30-99 lines. label Mar 29, 2021
@electronjoe electronjoe changed the title [CI] Add CodeCov.io flags to disambiguate coverage reports [CI] Add CodeCov.io flags to disambiguate reports Mar 29, 2021
We are observing erratic coverage reporting as CodeCov.io wants every git hash being reported to have complete coverage statistics - which we are not doing for latency / CPU / cost reasons (e.g. if a PR does not touch c/c++ targets we are omtting the hour+ of testing and upload of coverage).  This PR attempts to use per-coverage-domain tags to alleviate the issue.

The strategy is to apply a codecov.io project flag per CircleCI pipeline stage that pushes a coverage report - to ensure each can be run in isolation of any others.

Signed-off-by: Scott Harrison Moeller <smoeller@fb.com>
Copy link
Collaborator

@tmdzk tmdzk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codecov
Copy link

codecov bot commented Mar 29, 2021

Codecov Report

Merging #5788 (11d5bcd) into master (cd6221f) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #5788   +/-   ##
=======================================
  Coverage   37.10%   37.10%           
=======================================
  Files         984      984           
  Lines       89393    89393           
  Branches     1302     1302           
=======================================
  Hits        33173    33173           
  Misses      52926    52926           
  Partials     3294     3294           
Flag Coverage Δ
c_cpp 8.89% <ø> (?)
cloud_lint 66.33% <ø> (?)
feg-lint 56.18% <ø> (?)
lte-test 72.98% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd6221f...11d5bcd. Read the comment docs.

@electronjoe
Copy link
Member Author

Looks like it's working - I am seeing CodeCov per-project coverage stats that seem sane. Lets commit!

@electronjoe electronjoe reopened this Mar 29, 2021
@electronjoe electronjoe enabled auto-merge (squash) March 29, 2021 16:32
@electronjoe electronjoe merged commit 9f9ef67 into magma:master Mar 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/M Denotes a PR that changes 30-99 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants