Parallel tests

[boyanpenkov]

WIP

DONTMERGE

Here, I use x-dist to run the tests in parallel over multiple cores, which should speed things up there.

One issue that emerged is that there were race conditions in the tests. I addressed this in part by putting all the conflicting ones in a a new file -- test_serial -- which then gets run serially, so the speedup is not purely over all the cores at present.

Many of the race conditions were due to writing to the temp bib file on dist directly -- I'd be glad to try to address this using pytest fixtures, which can be instantiated in parallel.

What's more interesting is that having to do this by hand showed that a few tests were note being identified at all due to namespace collisions, and were not being run, so the test count went up by a few for free.

[boyanpenkov]

Note that this assumes #79 went in; if not, I can retract this.

[boyanpenkov]

Just an observation for now -- this fails if you run with tox --parallel to try to do all the envs at the same time, for the same fundamental reason -- trying to write to the same bibtex files in the same places.

Also, CI is still failing since the CI changes in #79 are not yet reflected here.

[boyanpenkov]

Note that I see that test_extract fails occasionally on py312 only, due to a file not being downloaded in downloadedpapers -- so there's another race condition, since this passes on the conda-tox branch. I'll look at this if we go forward here.

[boyanpenkov]

@perrette Another way I can do this is marking the relevant ones with --dist loadgroup till they stop failing, and then serializing the writes to disk files. This has the advantage of preserving the current test_* file structure explicitly, instead of my suggested temporary step of moving it into one file here...

Let me know?

[boyanpenkov]

@perrette Similarly here, there's a couple small test improvements that I can submit over time separately -- mainly stuff that flake8 picks up, but let me know if you're concerned about the state of the test suite here. Moving to https://docs.pytest.org/en/stable/explanation/fixtures.html would help things be more orthogonal, but your comment is spot on "The test below were written first. There are not systematic but they have the advantage to exist."

let me know...

[boyanpenkov]

Given the state of affairs in #79, can very likely close this.