Port analysis module to pytest

[utkbansal]

Fixes #

Changes made in this Pull Request:

PR Checklist

• Tests?
• Docs?
• CHANGELOG updated?
• Issue raised/referenced?

[utkbansal]

There is some unusual stuff going on in the TestHydrogenBondAnalysisChecking class. Can someone explain?

[richardjgowers]

@utkbansal not sure, but it might be because of self.universe being shared? Try replacing _setUp with a fixture, and make the _run method not operate on self variables

[kain88-de]

@utkbansal looks like you have to replace assertRaises and assert_warns with pytest equivalents.

[utkbansal]

@richardjgowers @kain88-de What should I do with this setup?

class TestEncoreImportWarnings(object):
    def setUp(self):
        # clear cache of encore module
        for mod in list(sys.modules):  # list as we're changing as we iterate
            if 'encore' in mod:
                sys.modules.pop(mod, None)

    @block_import('sklearn')
    def _check_sklearn_import_warns(self, package):
        warnings.simplefilter('always')
        assert_warns(ImportWarning, importlib.import_module, package)

    def test_import_warnings(self):
        for pkg in (
                'MDAnalysis.analysis.encore.dimensionality_reduction.DimensionalityReductionMethod',
                'MDAnalysis.analysis.encore.clustering.ClusteringMethod',
        ):
            yield self._check_sklearn_import_warns, pkg

This isn't something that a fixture is used for. This literally is some setup stuff. One option is to name this method setup and call it in every method. Any other non-hacky solutions?

[jbarnoud]

I did not test, but you should be able to do something like:

@pytest.fixture()
def remove_encore_module():
    # clear cache of encore module
    for mod in list(sys.modules):  # list as we're changing as we iterate
        if 'encore' in mod:
            sys.modules.pop(mod, None)

@pytest.mark.usefixtures('remove_encore_module')
class TestEncoreImportWarnings(TestCase):
    ...

From what I understand, the @pytest.mark.usefixtures decorator make sure the fixture is called but do not gives you access to the result of the fixture directly. Since the fixture returns nothing (None actually), it does not matter that you cannot access it; but you do not polute your tests with a useless argument to call the fixture.

Once again, I did not test so I might be wrong.

[utkbansal]

TestHydrogenBondAnalysisChecking is ultra-confusing. Other than that, everything else works.

[jbarnoud]

This changes the behaviour of the test. Instead of having 2 tests that can fail individually, there is only one test that has two consecutive asserts. It is fine for now, but please add a comment as it may be forgotten when you will move to proper pytest idioms.

[jbarnoud]

The comment is still missing here.

[jbarnoud]

The comment is bellow. I missed it.

[jbarnoud]

TestHydrogenBondAnalysisChecking is ultra-confusing

It is confusing indeed, but what do you find confusing there?

[jbarnoud]

Please prioritize this PR so we can get rid of nose.

[orbeckst]

Smaller things in the comments.

For the problem with test_hbonds.py I suggest you run just those tests locally. Looking at https://travis-ci.org/MDAnalysis/mdanalysis/jobs/251523905#L1441 some failures seem to come from something that pytest does:

E           AttributeError: 'TestHydrogenBondAnalysisChecking' object has no attribute 'universe'

Puzzling, because I don't think you touched TestHydrogenBondAnalysisChecking.

For the cryptic failures such as https://travis-ci.org/MDAnalysis/mdanalysis/jobs/251523905#L1302 add more details to the test output

try:
    ...
except Exception as err:
   raise AssertionError("... blabla failed. Exception: {}".format(err))

or similar. Then we might get a better idea where the problem lies. The hbond tests are admittedly not very good and just test that the code runs somehow but at least that should work. It would be better if they checked more specific things but that's outside the scope of your PR.

[orbeckst]

also remove nose from package/setup.py and testsuite/setup.py — yay, major step forward in your project: getting rid of nose.

[orbeckst]

And remove testsuite/mda_nosetests.py.

[orbeckst]

@mnmelo @jbarnoud @kain88-de @richardjgowers : should the nose-specific plugins also be removed at this time? We have to recreate the plugins for pytest or replace with native pytest plugins anyway,

[kain88-de]

@orbeckst these changes should be in a separate PR. The plugins can pretty much all go. There is the openfile plugin but otherwise pytest comes with better alternatives

[orbeckst]

remove use of mda_nosetests

[jbarnoud]

I agree with @orbeckst. With NOSE_TEST_LIST being empty, the call to mda_nosetests becomes buggy and should be removed. Removing the dependencies to nose is out of scope, though.

[orbeckst]

@utkbansal I assume the goal of this PR is to make it run under pytest and then you will do a second round adapting it to pytest style? Is this correct? (Would be fine with me.)

[kain88-de]

yes that idea is correct

[orbeckst]

remove DCD skipif, we have full DCD support now

[kain88-de]

better in a second PR

[orbeckst]

ok

[orbeckst]

good!

[orbeckst]

remove DCD skipif

[orbeckst]

separate PR, as @kain88-de says

[kain88-de]

@utkbansal You should prioritize this PR. The other PR's are nice but not as important as this one. You still haven't told us your exact problem.

[jbarnoud]

This change the behaviour and transforms a test generator into a succession of assertions in a single test. Please, add a comment so you can restore the initial behaviour latter.

[utkbansal]

I am still working on it. This isn't final.

[jbarnoud]

Same here. Add a comment.

[jbarnoud]

Idem. Comment.

[utkbansal]

@kain88-de @richardjgowers @jbarnoud This is the best hack I could come up with. Review please.

[mnmelo]

Must this become a list? Won't pytest.mark.parametrize accept the itertools generator object?

[kain88-de]

even if this has to be a list. Why not list(itertools.product(...))

[kain88-de]

The iterator still works just fine.

import pytest
import itertools

@pytest.mark.parametrize('a, b', itertools.product(range(3), range(3)))
def test_d(a, b):
    assert True

[mnmelo]

I'm not getting the rationale behind the two build_parameters. Is it to have the option of eventually supplying different lists of test args to the two tests?

[mnmelo]

Could you instead define protein, nothing and the itertools.product generator as class attributes of TestHydrogenBondAnalysisChecking? This way the the itertools.product will be available when the decorator is used, and there won't be any need for the build_parameters functions.

[utkbansal]

Can't access self in @pytest.mark.parametrize

[utkbansal]

I've already removed build_parameters()

[mnmelo]

Class attrs don't need to be accessed via self when in the class body. They're already in the namespace. Anyway, I think the solution of having the iterator call within the decorator is the best, as it clearer what's happening and allows customization in place.

[utkbansal]

So is the latest change ok?

I've done this

@pytest.mark.parametrize('s1, s2, s1type', itertools.product(
        ("protein", "resname ALA and not backbone"),
        ("protein", "resname ALA and not backbone"),
        ("donor", "acceptor", "both"),
    ))

[utkbansal]

nose is messing up now. It's trying to run everything.

[utkbansal]

@kain88-de @richardjgowers @jbarnoud The build technically passes because there are no errors in the pytest part.

[richardjgowers]

Yeah tests look ok now, the pytest half of things works. I'll let @orbeckst and @jbarnoud check their concerns

[utkbansal]

@richardjgowers So would be able to merge this as is(once everyone confirms the changes)? I'm already working on #1480 to remove nose specific stuff.

[jbarnoud]

Still a few comments, mostly about hbonds.

Also, I would like you to remove the calls to mda_nosetest in this PR to avoid breaking develop. Please do so on a different commit.

Finally, you could squash your commits from today (starting from "Quick fix"). Keep the commit that remove the calls to mda_nosetest separate.

[jbarnoud]

The comment is still missing here.

[jbarnoud]

The way the test is written now, what is the point of having the nested function?

[utkbansal]

Nothing really. I can pull that out.

[jbarnoud]

I do not like having functions named run_HBA and run_HBA2. It was OK that they had the same name when they were nested because the parent function was giving the context. Now the names do not tell how the two functions differ from each other. run_HBA_no_update and run_HBA_update would be better names, other names may work as well.

[jbarnoud]

I agree with @orbeckst. With NOSE_TEST_LIST being empty, the call to mda_nosetests becomes buggy and should be removed. Removing the dependencies to nose is out of scope, though.

[utkbansal]

@jbarnoud If I understand correctly, I have to squash all my commits now and then have one more commit that removes mda_nosetests . In the end, one commit for porting analysis module and one for removing mda_nosetests

[jbarnoud]

All your commits until the one titled "Quick fix" are logical units. I would squash them but I do not mind having them separate. From "Quick fix" and latter, the commits are iterative fixes that should be squashed. The removal of mda_nosetest must be on a separate commit.

[utkbansal]

It's hard to believe, but the full build completed in 16.5 mins.

[utkbansal]

@jbarnoud All good?

[utkbansal]

The build passed! Plus a minor increase in coverage.

[kain88-de]

@orbeckst do you have any more comments? I would like to merge this.