Quality of Life: Make codecov less meticulous

[Corvince]

If you currently look at the open PRs there is not a single PR that passes all our checks. For most PRs I found that at least Codecov is failing. I think the current settings are a bit too agressive. For example take #1964 . In this This PR restructures some code. Everything is still covered, but because its a single line of code more the overall % of covered lines decreased by 0.07% and the check failed.

In this PR I set the target value for the project to 80% with a 1% threshold. We are currently just above 79% so if this decreases significantly the checks will fail.

I also set it to ignore experimental folders (jupyterviz)

And finally I disabled comments on PR. This might be debatable, but I found myself just ignoring those comments after having seen too many. For me its very similar for checks. I don't notice failed checks sometimes because I am so used to the red cross on Mesa. I hope this will get better with this PR. Additionally it would be great if we could setup pre-commit autoupdating PRs, but this would be in contrast to the ideas proposed in #1963

[Corvince]

In this PR I set the target value for the project to 80% with a 1% threshold. We are currently just above 79% so if this decreases significantly the checks will fail.

Maybe I should change this. With experimental ignored we are actually at 90%

[quaquel]

I would like you to include experimental stuff in the code coverage to encourage proper quality assurance. I also find it often helpful to understand better what a given piece of code is trying to achieve. Beyond that, I fully agree with your motivation for making the threshold less strict.

I am all in favor of using pre-commit for stuff like a ruff. It is a simple quality-of-life improvement that makes committing and creating PRs just that bit easier. I am still gathering my thoughts on #1963 and will reply there later this week.

[EwoutH]

Agreed on the sentiment and direction, indifferent on the exact implementation.

The experimental space is a bit of a free for all, so I think we don't need such strict standards, but experimental features outside the experimental space are still required coverage. So I think this PR is good as is, but I also wouldn't object against it including the experimental space.

[Corvince]

I would like you to include experimental stuff in the code coverage to encourage proper quality assurance. I also find it often helpful to understand better what a given piece of code is trying to achieve. Beyond that, I fully agree with your motivation for making the threshold less strict.

Thought about this the last days. I think this is again a matter of how you define experimental. Currently only the JupyterViz is in the ignored folder. I decided to keep this PR for two reasons

• The API there is very likely to change so writing tests at this point is very tedious
• Writing frontend unit tests isn't super useful. Yes, you can test some things, but for most things End-to-End tests make much more sense - but they are unrelated to code coverage.

[quaquel]

Fair enough.