pytest 4.1.0 may have broken pytest-cov

[canyon289]

In the ArviZ project we started getting test failures when using pytest-cov 2.6.0 with pytest 4.1.0 which was released yesterday.

TravisCI UUID 476024505
https://travis-ci.org/arviz-devs/arviz/jobs/476024505#L2089
https://travis-ci.org/arviz-devs/arviz/jobs/476024505#L2795

Pinning pytest back to 4.0.2 seems to have worked locally. When I get a TravisCI build that shows this I'll post links here as well.

Please forgive me for not having a minimally reproducible example. If this description isn't enough I can follow up with more information.

And thank you for maintaining this plugin. Its absolutely fantastic

[hugovk]

I think this has been fixed by #253.

[canyon289]

Ah great! Thank you, we'll wait for it to hit pypi.

Really appreciate this. Feel free to close

[rafalkasa]

I have the same issue on my project any news when the new version of pytest-cov will be released?

[Kilo59]

I don't know if this is the same issue but according to the pytest Plugins Compatibility site the latest release of pytest-cov is broken/not compatible on 2.7, 3.6, 3.7.

Personally I'm only having trouble on 3.6 and 2.7, but not 3.7... which is wierd.

[ionelmc]

@Kilo59 so it looks like your travis issue is that you have too old pytest there while pytest-cov needs 3.6+. You already have pytest installed there for some reason and pip doesn't upgrade it. Dunno why that happens, but we have the version constraint so I suggest you go complain on pip's bugtracker or use a workaround (install everything in a clean environment, or use --upgrade --ignore-installed or similar).

About http://plugincompat.herokuapp.com/ showing pytest-cov as failing - it's because pytest-cov's doesn't have a simple env list like py27,py34,py35,py36. Maybe @nicoddemus has some thoughts about how it can be fixed (I personally don't think it should handle CI for each project but rather only show an external CI badge and package metadata).

@canyon289 @rafalkasa 2.6.1 was released.

[canyon289]

@ionelmc Thank you!

[nicoddemus]

About http://plugincompat.herokuapp.com showing pytest-cov as failing - it's because pytest-cov's doesn't have a simple env list like py27,py34,py35,py36.

Yep, the plugin compatibility page should be taken with grain of salt, not as an authoritative compatibility report.

Perhaps we should list the envs explicitly instead of trying to always use py27 and py37... 🤔

[ionelmc]

Perhaps we should list the envs explicitly instead of trying to always use py27 and py37... 🤔

But then you practically reimplement every plugin's CI, but badly. Reminiscent of Greenspun's tenth rule of programming :>

[nicoddemus]

Good point. The initial idea was to run using ourselves because the objective was to test compatibility with different pytest versions, but this has been proven really fragile.