CI: use nightly wheels

[bsipocz]

This PR does a few minor CI improvements:

• switches to use "nightly" wheels for dev versions rather than building from source
• opt-in to also include testing with the dev numpy
• makes sure that we actually test with the mandatory dependencies only (previously all dependencies were always installed)
• run coverage test after a test suite run rather than after running the linter (as that one shouldn't install dependencies)
• minor cleanups to ensure the relevant versions are printed in the test header

[pllim]

Stuart proposed another way to do this over at astropy/astropy#13614

[bsipocz]

Hmm, interesting. That was more like the first approach we had, and then we switched over to having these individual pip installs. I think Stuart's version is cleaner in the case when you have multiple packages. However, here we only use the scipy-wheels-nightly for numpy, and something else for astropy, and don't install other packages. So I would say we should keep it as a one-liner like this rather than adding extras.

[Cadair]

There's very little to choose between them, so use your favourite, but sunpy uses multiple index URLs with the same pattern https://github.com/sunpy/sunpy/blob/main/tox.ini#L34

[pllim]

Is this still needed? I think astropy stopped testing this a long time ago.

[bsipocz]

yes, I suppose it can go.

[pllim]

At a glance, LGTM.

You would want to inspect the logs carefully to make sure each job is running on the versions you think they are running on. Nice work!

[andamian]

Great work @bsipocz - the CI is faster by an order of 3 or 4. Awesome!

Coverage fails to upload. Wondering if it's because of the missing Graphviz dependency

The point in the description ("makes sure that we actually test with the mandatory dependencies only (previously all dependencies were always installed)") - I couldn't figure out where/how it's addressed

The rest looks good to me. Thanks!

[bsipocz]

The point in the description ("makes sure that we actually test with the mandatory dependencies only (previously all dependencies were always installed)") - I couldn't figure out where/how it's addressed

haha, it actually helped me to realize that there were some syntax mistakes done while clearing up tox. So, I expect things won't be as fast now.

[codecov]

Codecov Report

Merging #353 (488072f) into main (82bb4a4) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #353      +/-   ##
==========================================
- Coverage   78.28%   78.27%   -0.01%     
==========================================
  Files          46       46              
  Lines        5494     5497       +3     
==========================================
+ Hits         4301     4303       +2     
- Misses       1193     1194       +1     

Impacted Files	Coverage Δ
pyvo/__init__.py	100.00% <ø> (ø)
pyvo/dal/params.py	86.38% <0.00%> (-0.34%)	⬇️
pyvo/dal/query.py	85.16% <0.00%> (+0.03%)	⬆️
pyvo/dal/tap.py	69.49% <0.00%> (+0.08%)	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[bsipocz]

This should now fix and close #354. Note that mentioning mimeparse has been removed from docs and config as I see it is in fact not being used (and hasn't been release either in the past ~9 years)

[bsipocz]

OK, so picked up versions seem to be correct, this now only needs an approval and ready to go in.

[andamian]

It looks good to me, but now that the CI speed gains seem to be gone it begs the question of what is the point of using wheels (and add an extra dependency on anaconda and the process that publishes the wheels) in this particular case?

To be clear: the cleanup done here is still most welcome.

[pllim]

CI speed gains seem to be gone

What do you mean? The dev job now only takes 1 min. The dev job in main takes 3 mins.

[andamian]

CI speed gains seem to be gone

What do you mean? The dev job now only takes 1 min. The dev job in main takes 3 mins.

I might compare apples with oranges here but the total execution time improved from 5.27min to 5.20min. Certainly not the big jump that I saw initially.

[bsipocz]

It looks good to me, but now that the CI speed gains seem to be gone it begs the question of what is the point of using wheels (and add an extra dependency on anaconda and the process that publishes the wheels) in this particular case?

The whole point here is to avoid building astropy and numpy from source, yet test with the dev version.
Anaconda is only providing the storage location, they are not at all involved in the process of producing and publishing these wheels. So the system is pretty reliable, as well as the standard process by the big libraries. So I wouldn't worry about this.

As for CI speed improvement is gone, yep, there was a syntax mistake so the tests were not run in fact, so no wonder the jobs were faster 😊

[pllim]

improved from 5.27min to 5.20min

That makes sense to me. Building from source isn't that long to begin with. A bulk of the actual run time is setting up the test env, running tests, and then the reporting.

[bsipocz]

(oh, and I'm not sure where that 5 mins is coming from, it has some parallelization included already. If you click on it, it's 11 mins vs 16-13 min of total runtime.)

[bsipocz]

improved from 5.27min to 5.20min

That makes sense to me. Building from source isn't that long to begin with. A bulk of the actual run time is setting up the test env, running tests, and then the reporting.

Note that we also use numpy-dev atm, and that shouldn't not be built from source.

[andamian]

All good! It was probably just that the initial (buggy) version set the expectations too high. :-). Thanks for the improvements.