Add support for classes with pytest 7

[ConorMacBride]

Since pytest 7 was released pytest-mpl has not been running tests inside classes. This means that figure tests inside classes are always being reported as passing. In this PR, I have added additional tests that ensure figure tests inside classes can fail correctly, and I have also added new testing jobs for older versions of pytest as well as the development version.

• Find the bug and fix it
• Make a bugfix release of pytest-mpl

Solution

Instead modifying the test function itself by wrapping it in a function which runs the tests, this PR uses pytest hooks to intercept the generated figure and then run the tests. This should be a more robust approach that doesn't need as many special cases to be hardcoded (such as classes and support for setup_method etc.).

Previously we updated the test function (to a version of itself but wrapped in the testing code) inside the pytest_runtest_setup hook. However, now we work inside the pytest_runtest_call hook and override pytest's default pytest_pyfunc_call hook:

1. A test starts
2. Pytest calls our pytest_runtest_call hook
3. It runs until yield
4. Pytest then calls our pytest_pyfunc_call hook which runs the test function, intercepts the returned figure, and saves it as an attribute (self.result)
5. Pytest then continues after the yield inside our pytest_runtest_call hook
6. It accesses the figure from the attribute self.result
7. It tests the figure

I have also combined the get_marker and get_compare functions/methods to simplify these.

[lgtm-com]

This pull request introduces 2 alerts when merging 83359e6 into b8bcfe1 - view on LGTM.com

new alerts:

• 2 for Unused import

[ConorMacBride]

To ensure pytest-mpl works across different versions of pytest, I have added some new tests:

- linux: py310-test-mpl35-pytestdev
- linux: py310-test-mpl35-pytest62
- linux: py38-test-mpl35-pytest54

Since pytest doesn't seem to release bug fixes for previous major/minor versions, I have just added tests for the latest release for each major version from the past ~2 years. (We should probably set up a weekly cron for the tests also?)

I've also added a test using the development version of pytest. (This uncovered that pytest are going to start warning and eventually erroring if a test returns anything other than None, however provided pytest-mpl is installed, this warning will not be shown.)

[ConorMacBride]

I don't think we need to support this branch anymore.

[ConorMacBride]

I've also updated the generate_test_name method to include the class name in the test name (if test is inside a class): 19dac01

This is technically a bugfix as the following should be able to generate three different baselines, but currently without the class name they overwrite each other and only the final test will pass when tested:

class TestA:
    @pytest.mark.mpl_image_compare
    def test_a(self): ...
class TestB:
    @pytest.mark.mpl_image_compare
    def test_a(self): ...
@pytest.mark.mpl_image_compare
def test_a(self): ...

This is a breaking change for figure tests within classes only, and the following will need to be done:

• Hash library test names will need to be regenerated/updated to include the class name.
• Iff the undocumented mpl-use-full-test-name ini option is enabled, the the baseline images will need to be regenerated, or have their filename updated to include the class name.

Thought I would include this here while we are fixing the classes.

[ConorMacBride]

@Cadair I ran the astropy and sunpy figure tests against this branch and pytest-mpl worked correctly, with most tests passing and some failing because of the different mpl version I used. Both produced a complete HTML report without any tests missing.

[Cadair]

The tests pass so it's probably fine 🤣