[WIP] Enable codecov

[pentschev]

Solves #2218.

cc @mrocklin @jakirkham @TomAugspurger

• Tests added / passed
• Passes flake8 dask

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@cb58d97). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #4631   +/-   ##
=========================================
  Coverage          ?   91.02%           
=========================================
  Files             ?       92           
  Lines             ?    17165           
  Branches          ?        0           
=========================================
  Hits              ?    15624           
  Misses            ?     1541           
  Partials          ?        0

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cb58d97...fa87b94. Read the comment docs.

[pentschev]

Looks like we have something here already. We should probably tweak what options we want to enable, and maybe decide on enforcement policy. For now I suggest we don't enforce coverage maintenance (i.e., that it's never negative) but provide this as a tool for developers to check how PRs affect the project, at least until we are comfortable that options we set suit our needs.

[jakirkham]

Generally seems ok to me.

Would slightly prefer if we could turn the comments off (to cutdown on noise). Also would be good to enable the status information so we can link to the details on their website as needed.

[pentschev]

Generally seems ok to me.

Would slightly prefer if we could turn the comments off (to cutdown on noise). Also would be good to enable the status information so we can link to the details on their website as needed.

Agreed.

[pentschev]

I think to get codecov statuses we need to install its Codecov Github app. Could someone with permissions do that?

[mrocklin]

Done, I think

[pentschev]

We have status now, thanks @mrocklin for installing the app. I've attempted to add coverage for each submodule, but they all seem to redirect to the same place, I'm not sure if that's because there's no report (we have to merge to get it first) or something wrong with the configuration.

Either way, would you say we should have codecov status for each submodule as different projects or just a single one? I'm personally fine with either.

[jakirkham]

Thanks Matt!

No strong feelings. We might be ok with just one report near term. However if that's difficult to do, wouldn't worry about it.

[pentschev]

On the contrary, it's actually easier. I just thought it would be cool to see them separately, but I'm not sure that worked. For now I'm gonna revert it into a single project then.

…

On Thu, Mar 28, 2019, 8:04 AM jakirkham ***@***.***> wrote: Thanks Matt! No strong feelings. We might be ok with just one report near term. However if that's difficult to do, wouldn't worry about it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4631 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEMcptPLUOI2S1CPZ1SojCR-I4VvGREWks5vbGlvgaJpZM4cLxRv> .

[pentschev]

Ok, we now have the status for a single project setup. Unless someone already knows about other options that we may want, I would suggest we merge this PR and start looking at codecov comparisons after this to improve it further, since I don't think we can see the comparisons before it gets merged and creates a baseline report.

[mrocklin]

OK to merge?

[jrbourbeau]

+1 from me