Use mpirun when testing MPI

[aragilar]

This provides more freedom to specify HDF5 locations (which I needed for testing the MPI builds), and runs the tests under MPI (apart from those known to not work, e.g. those using subprocess). I fixed the existing MPI tests, and added some docs about how to test with MPI.

This depends on https://pypi.org/project/pytest-mpi/, which I created because it seems that the conftest.py must the at the root level for cli args to work (see https://stackoverflow.com/questions/52447767/using-pytest-addoptions-in-a-non-root-conftest-py), I'm happy to move it under the h5py org and/or give people access (on github, pypi and other services) if people think that's appropriate (I plan on adding docs and tests, happy to restructure it also, I basically used an existing pytest plugin I wrote as a template).

Before this gets merged, can people run this on their systems (and if report any lockups, the tests should run very fast, so if you're waiting a while, there's probably a problem).

#1206 will need to be rebased on this (let me know if there are more features we should expose with pytest-mpi).

[takluyver]

I don't have time to look at this in detail right now, but this sounds excellent - having a decent testing infrastructure for MPI will make things like #1206 much simpler to address.

For the plugin: I think it's fine to add it to the h5py org if you'd like to maintain it collaboratively. But we can also depend on your project if you'd rather keep it separate.

[codecov]

Codecov Report

Merging #1255 into master will decrease coverage by 0.71%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1255      +/-   ##
==========================================
- Coverage   84.95%   84.24%   -0.72%     
==========================================
  Files          17       17              
  Lines        2107     2107              
==========================================
- Hits         1790     1775      -15     
- Misses        317      332      +15

Impacted Files	Coverage Δ
h5py/_hl/files.py	91.26% <0%> (-4.37%)	⬇️
h5py/_hl/dataset.py	87.23% <0%> (-1.26%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eede146...c71ca9d. Read the comment docs.

[aragilar]

I've switched back to using the system HDF5 (on azure now, rather than travis). I won't do any more experimentation on this branch, I'll leave getting a newly-built version of HDF5 to a different PR/branch.

[takluyver]

It looks like the tests crashed immediately while trying to import h5py, but then sat until they timed out after an hour. Before making the tests work, can we ensure that it stops correctly on failure?

[aragilar]

I've dropped the CI changes from this, so hopefully the tests now pass (leaving the current MPI tests on travis). I'll look at expanding the CI factors the MPI tests run on in a different PR.

[takluyver]

The test changes here look good to me; thanks.