New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port core module to pytest #1443

Merged
merged 26 commits into from Jul 2, 2017

Conversation

Projects
None yet
4 participants
@utkbansal
Member

utkbansal commented Jun 28, 2017

Fixes #

Changes made in this Pull Request:

PR Checklist

  • Tests?
  • Docs?
  • CHANGELOG updated?
  • Issue raised/referenced?

utkbansal added some commits Jun 28, 2017

@utkbansal utkbansal changed the title from Port core module to pytest to [WIP]Port core module to pytest Jun 28, 2017

utkbansal added some commits Jun 28, 2017

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jun 28, 2017

I hope this fixes it.

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jun 29, 2017

I'm pretty sure there is something odd going on with yield based tests. I'm trying to isolate the issue and hopefully make a smaller example from it.

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jun 29, 2017

@richardjgowers @kain88-de I think we have an issue here.

yield-based methods don’t support setup properly because the setup method is always called in the same class instance. There are no plans to fix this currently because yield-tests are deprecated in pytest 3.0, with pytest.mark.parametrize being the recommended alternative.

Source: https://docs.pytest.org/en/latest/nose.html#unsupported-idioms-known-issues

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jun 29, 2017

Oh crap, pytest shows me that the tests are collected, but they aren't being executed, they just pass.

Try running this pytest core/test_atomselections.py::TestImplicitOr --collect-only
Output is

<Module 'MDAnalysisTests/core/test_atomselections.py'>
  <UnitTestCase 'TestImplicitOr'>
    <TestCaseFunction 'test_range_selections'>
    <TestCaseFunction 'test_string_selections'>

But neither of the two methods are executed! They just show up as pass. I can confirm this because I've tried adding print statements/using the debugger, but its not executed.

The only solution is to get rid of yeild based tests and use parameterize.

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jun 29, 2017

@richardjgowers @kain88-de

What's wrong with the following conversion to parameterize?

    def test_string_selections(self):
        for ref, sel in (
                ('name NameABA or name NameACA or name NameADA',
                 'name NameABA NameACA NameADA'),
                ('type TypeE or type TypeD or type TypeB',
                 'type TypeE TypeD TypeB'),
                ('resname RsC or resname RsY', 'resname RsC RsY'),
                ('name NameAB* or name NameACC', 'name NameAB* NameACC'),
                ('(name NameABC or name NameABB) and (resname RsD or resname RsF)',
                 'name NameABC NameABB and resname RsD RsF'),
                ('segid SegA or segid SegC', 'segid SegA SegC'),
        ):
            yield self._check_sels, ref, sel

    @pytest.mark.parametrize('ref, sel', (
        ('name NameABA or name NameACA or name NameADA',
         'name NameABA NameACA NameADA'),
        ('type TypeE or type TypeD or type TypeB',
         'type TypeE TypeD TypeB'),
        ('resname RsC or resname RsY', 'resname RsC RsY'),
        ('name NameAB* or name NameACC', 'name NameAB* NameACC'),
        ('(name NameABC or name NameABB) and (resname RsD or resname RsF)',
         'name NameABC NameABB and resname RsD RsF'),
        ('segid SegA or segid SegC', 'segid SegA SegC'),
    ))
    def test_string_selections_py(self, ref, sel):
        self._check_sels(ref, sel)

I am getting an error TypeError: test_string_selections_py() takes exactly 3 arguments (1 given)

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jun 29, 2017

Oh man, more mess. 😭

@pytest.mark.parametrize doesn't work if I inherit from TestCase

Edit
Source: https://stackoverflow.com/questions/18182251/does-pytest-parametrized-test-work-with-unittest-class-based-tests

@richardjgowers

This comment has been minimized.

Member

richardjgowers commented Jun 29, 2017

@utkbansal we're moving away from TestCase and yield anyway, so it sounds like you'll have to port this 100% to pytest rather than the 50% you've been able to do with other tests. In the example you've copied, it should be pretty easy to make u from setUp a fixture, then just pass that to the thing you've written.

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jun 29, 2017

@richardjgowers Yeah, I'm worried that this will take some additional time and will mess up with my timeline. Though I have to do this work anyway, but the time taken on Phase 1 gets extended.

@kain88-de

This comment has been minimized.

Member

kain88-de commented Jun 29, 2017

@utkbansal you might want to work on analysis then before you complete this. But yeah you have to do the work anyway. And since this is one of the last tests still on nose it isn't that bad.

@kain88-de

This comment has been minimized.

Member

kain88-de commented Jun 29, 2017

Also a quick hack would be

def test_string_selections_py(self):
     stuff = (('name NameABA or name NameACA or name NameADA',  'name NameABA NameACA NameADA'),
     ('type TypeE or type TypeD or type TypeB', 'type TypeE TypeD TypeB'),
        ('resname RsC or resname RsY', 'resname RsC RsY'),
        ('name NameAB* or name NameACC', 'name NameAB* NameACC'),
        ('(name NameABC or name NameABB) and (resname RsD or resname RsF)',
         'name NameABC NameABB and resname RsD RsF'),
        ('segid SegA or segid SegC', 'segid SegA SegC'))
    for ref, sel in stuff:
        self._check_sels(ref, sel)

I don't recommend it though because this has to be converted to proper pytests anyway.

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jun 29, 2017

Update: @dec.skipif doesn't play well if the fixture is marked as @staticmethod

@kain88-de

This comment has been minimized.

Member

kain88-de commented Jun 29, 2017

('prop mass < 4.0 hello', universe), # unused token
('prop mass > 10. and group this', universe), # missing group
('prop mass > 10. and fullgroup this', universe), # missing fullgroup
], indirect=['universe'])

This comment has been minimized.

@kain88-de

kain88-de Jun 30, 2017

Member

what does the indirect= do? Is that documented somewhere?

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jun 30, 2017

]:
yield self.selection_fail, selstr
@pytest.mark.parametrize('selstr, universe', [

This comment has been minimized.

@richardjgowers

richardjgowers Jun 30, 2017

Member

why is universe being parametrized here?

This comment has been minimized.

@utkbansal

utkbansal Jun 30, 2017

Member

Because the function doesn't get it directly from the fixture. Have a look at the link I posted above.

This comment has been minimized.

@richardjgowers

richardjgowers Jun 30, 2017

Member

Right, but why can't we just use the Universe fixture above? ie just

@parametrize('selstr', [etc etc])
def test(self, selstr, universe):

This comment has been minimized.

@utkbansal

utkbansal Jun 30, 2017

Member

I tried that, it didn't work. I don't have a reason to why it didn't work. I'll try to look it up.

This comment has been minimized.

@kain88-de

kain88-de Jul 1, 2017

Member

The suggestion of @richardjgowers work for me locally

def selection_fail(self, selstr):
assert_raises(SelectionError, self.u.select_atoms,
def selection_fail(self, selstr, universe):

This comment has been minimized.

@richardjgowers

richardjgowers Jun 30, 2017

Member

now we're not using yield (and having to provide a function to call for each test), we can collapse these functions into the test function itself

sel.format(typ=seltype))
class TestICodeSelection(object):
@pytest.mark.parametrize('ref, sel, universe',[

This comment has been minimized.

@richardjgowers

richardjgowers Jun 30, 2017

Member

same as above, why is Universe in the parametrize?

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jun 30, 2017

The drop in coverage should be fixed now, hopefully.

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jun 30, 2017

@richardjgowers @kain88-de I've got the coverage back to normal. Could you please review. Then I can do all the cleanup in one go.

@@ -497,125 +509,134 @@ def choosemeth(sel, meth, periodic):
return sel
def _check_around(self, meth, periodic):
sel = Parser.parse('around 5.0 resid 1', self.u.atoms)
@pytest.mark.parametrize('u, meth, periodic', [

This comment has been minimized.

@richardjgowers

richardjgowers Jun 30, 2017

Member

If you change the u fixture above to a staticmethod, you don't have to parametrize around it here.

ie

@staticmethod
@pytest.fixture
def u():

@pytest.mark.parametrize('meth, periodic', [])
def test_around(self, u, meth, periodic):

will work

This comment has been minimized.

@utkbansal

utkbansal Jul 1, 2017

Member

Oh yes, when I was trying I had a dec.skipif and that doesn't work well with static methods. That's why I got different results.

@richardjgowers

needs the indirects removing first

('improper', u)
], indirect=['u'])
def test_VEs(self, btype, u):
self._check_VE(btype, u)

This comment has been minimized.

@kain88-de

kain88-de Jul 1, 2017

Member

can you directly include the _check_VE function here.

# 'prop mass > 10. and group this', # missing group
# 'prop mass > 10. and fullgroup this', # missing fullgroup
# ]:
# yield self.selection_fail, selstr

This comment has been minimized.

@kain88-de

kain88-de Jul 1, 2017

Member

this should be removed before the final merge

]:
yield self.selection_fail, selstr
@pytest.mark.parametrize('selstr, universe', [

This comment has been minimized.

@kain88-de

kain88-de Jul 1, 2017

Member

The suggestion of @richardjgowers work for me locally

utkbansal added some commits Jul 1, 2017

Refactor
* Get rid of the indirect argument
* Use @pytest.mark.skipif in place of @dec.skipif
def tearDown(self):
del self.u
@pytest.mark.skipif(parser_not_found('DCD'),'DCD parser not available. Are you using python 3?')

This comment has been minimized.

@richardjgowers

richardjgowers Jul 1, 2017

Member

Can you double check that marking fixtures like this works? Ie does this mark then get transferred onto tests which use this fixture?

This comment has been minimized.

@utkbansal

utkbansal Jul 1, 2017

Member

On my local system, I only have the minimal dependencies, all the test methods are skipped.

This comment has been minimized.

@kain88-de

kain88-de Jul 1, 2017

Member

You run on python3? These tests will always run on Python 2

This comment has been minimized.

@utkbansal

utkbansal Jul 1, 2017

Member

That's odd. I'm python 2.

This comment has been minimized.

@kain88-de

kain88-de Jul 1, 2017

Member

dcd is available on python 2 as a parser. Those tests should run for you. But since we soon have a python3 DCD parser I'm not to worried if the tests are skipped but they should be run on phython2.

This comment has been minimized.

@richardjgowers

richardjgowers Jul 2, 2017

Member

This should work on Python 2

This comment has been minimized.

@kain88-de

kain88-de Jul 2, 2017

Member

@utkbansal do these tests work under python 2 now?

This comment has been minimized.

@utkbansal

utkbansal Jul 2, 2017

Member

Yes, they work now.

@staticmethod
@pytest.fixture()
def u(self):
# TODO: This is dummy. Needs review!

This comment has been minimized.

@richardjgowers

richardjgowers Jul 1, 2017

Member

Is this still Todo, or is it Todone?

This comment has been minimized.

@utkbansal

utkbansal Jul 1, 2017

Member

Now that we don't need to pass universe to the parametrize call, we can get rid of this completely.

@orbeckst orbeckst added the testing label Jul 1, 2017

@utkbansal

This comment has been minimized.

Member

utkbansal commented Jul 2, 2017

@richardjgowers @kain88-de What else needs to be done?

def tearDown(self):
del self.u
@pytest.mark.skipif(parser_not_found('DCD'),'DCD parser not available. Are you using python 3?')

This comment has been minimized.

@kain88-de

kain88-de Jul 2, 2017

Member

@utkbansal do these tests work under python 2 now?

('{typ} 1-5 or {typ} 7:10 or {typ} 12', '{typ} 1-5 7:10 12'),
('{typ} 1 or {typ} 3 or {typ} 5:10', '{typ} 1 3 5:10'),
])
def test_range_selections(self, seltype, ref, sel, universe):

This comment has been minimized.

@kain88-de

kain88-de Jul 2, 2017

Member

how many tests are generated for this? Does it equal the number of tests the old double loop used in nose?

This comment has been minimized.

@utkbansal

utkbansal Jul 2, 2017

Member

18 test cases are generated as expected.

@kain88-de

This comment has been minimized.

Member

kain88-de commented Jul 2, 2017

@richardjgowers you should have a final look for the core module.

@richardjgowers

This comment has been minimized.

Member

richardjgowers commented Jul 2, 2017

Yep looks good

@utkbansal utkbansal changed the title from [WIP]Port core module to pytest to Port core module to pytest Jul 2, 2017

@kain88-de kain88-de merged commit b97f2ab into MDAnalysis:develop Jul 2, 2017

3 checks passed

QuantifiedCode 57 minor issues introduced.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage remained the same at 89.561%
Details

@utkbansal utkbansal referenced this pull request Jul 2, 2017

Merged

Fix coverage drop in coordinates module #1441

0 of 4 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment