Port core module to pytest by utkbansal · Pull Request #1443 · MDAnalysis/mdanalysis

utkbansal · 2017-06-28T13:53:49Z

Fixes #

Changes made in this Pull Request:

PR Checklist

Tests?
Docs?
CHANGELOG updated?
Issue raised/referenced?

utkbansal · 2017-06-28T16:55:04Z

I hope this fixes it.

utkbansal · 2017-06-29T12:46:01Z

I'm pretty sure there is something odd going on with yield based tests. I'm trying to isolate the issue and hopefully make a smaller example from it.

utkbansal · 2017-06-29T12:51:05Z

@richardjgowers @kain88-de I think we have an issue here.

yield-based methods don’t support setup properly because the setup method is always called in the same class instance. There are no plans to fix this currently because yield-tests are deprecated in pytest 3.0, with pytest.mark.parametrize being the recommended alternative.

Source: https://docs.pytest.org/en/latest/nose.html#unsupported-idioms-known-issues

utkbansal · 2017-06-29T12:57:51Z

Oh crap, pytest shows me that the tests are collected, but they aren't being executed, they just pass.

Try running this pytest core/test_atomselections.py::TestImplicitOr --collect-only
Output is

<Module 'MDAnalysisTests/core/test_atomselections.py'>
  <UnitTestCase 'TestImplicitOr'>
    <TestCaseFunction 'test_range_selections'>
    <TestCaseFunction 'test_string_selections'>

But neither of the two methods are executed! They just show up as pass. I can confirm this because I've tried adding print statements/using the debugger, but its not executed.

The only solution is to get rid of yeild based tests and use parameterize.

utkbansal · 2017-06-29T14:07:00Z

@richardjgowers @kain88-de

What's wrong with the following conversion to parameterize?

    def test_string_selections(self):
        for ref, sel in (
                ('name NameABA or name NameACA or name NameADA',
                 'name NameABA NameACA NameADA'),
                ('type TypeE or type TypeD or type TypeB',
                 'type TypeE TypeD TypeB'),
                ('resname RsC or resname RsY', 'resname RsC RsY'),
                ('name NameAB* or name NameACC', 'name NameAB* NameACC'),
                ('(name NameABC or name NameABB) and (resname RsD or resname RsF)',
                 'name NameABC NameABB and resname RsD RsF'),
                ('segid SegA or segid SegC', 'segid SegA SegC'),
        ):
            yield self._check_sels, ref, sel

    @pytest.mark.parametrize('ref, sel', (
        ('name NameABA or name NameACA or name NameADA',
         'name NameABA NameACA NameADA'),
        ('type TypeE or type TypeD or type TypeB',
         'type TypeE TypeD TypeB'),
        ('resname RsC or resname RsY', 'resname RsC RsY'),
        ('name NameAB* or name NameACC', 'name NameAB* NameACC'),
        ('(name NameABC or name NameABB) and (resname RsD or resname RsF)',
         'name NameABC NameABB and resname RsD RsF'),
        ('segid SegA or segid SegC', 'segid SegA SegC'),
    ))
    def test_string_selections_py(self, ref, sel):
        self._check_sels(ref, sel)

I am getting an error TypeError: test_string_selections_py() takes exactly 3 arguments (1 given)

utkbansal · 2017-06-29T14:15:53Z

Oh man, more mess. 😭

@pytest.mark.parametrize doesn't work if I inherit from TestCase

Edit
Source: https://stackoverflow.com/questions/18182251/does-pytest-parametrized-test-work-with-unittest-class-based-tests

richardjgowers · 2017-06-29T14:35:00Z

@utkbansal we're moving away from TestCase and yield anyway, so it sounds like you'll have to port this 100% to pytest rather than the 50% you've been able to do with other tests. In the example you've copied, it should be pretty easy to make u from setUp a fixture, then just pass that to the thing you've written.

utkbansal · 2017-06-29T14:37:39Z

@richardjgowers Yeah, I'm worried that this will take some additional time and will mess up with my timeline. Though I have to do this work anyway, but the time taken on Phase 1 gets extended.

kain88-de · 2017-06-29T14:47:19Z

@utkbansal you might want to work on analysis then before you complete this. But yeah you have to do the work anyway. And since this is one of the last tests still on nose it isn't that bad.

kain88-de · 2017-06-29T14:50:00Z

Also a quick hack would be

def test_string_selections_py(self):
     stuff = (('name NameABA or name NameACA or name NameADA',  'name NameABA NameACA NameADA'),
     ('type TypeE or type TypeD or type TypeB', 'type TypeE TypeD TypeB'),
        ('resname RsC or resname RsY', 'resname RsC RsY'),
        ('name NameAB* or name NameACC', 'name NameAB* NameACC'),
        ('(name NameABC or name NameABB) and (resname RsD or resname RsF)',
         'name NameABC NameABB and resname RsD RsF'),
        ('segid SegA or segid SegC', 'segid SegA SegC'))
    for ref, sel in stuff:
        self._check_sels(ref, sel)

I don't recommend it though because this has to be converted to proper pytests anyway.

utkbansal · 2017-06-29T19:48:25Z

Update: @dec.skipif doesn't play well if the fixture is marked as @staticmethod

kain88-de · 2017-06-29T19:59:51Z

@utkbansal use pytest.mark.skipif. Also where did you come across the skipif?

utkbansal · 2017-06-29T20:02:22Z

@kain88-de https://github.com/MDAnalysis/mdanalysis/pull/1443/files#diff-ab92738dd8fc8e62b290ddbdd5eb909dR142

kain88-de · 2017-06-29T20:04:47Z

@utkbansal this should work. https://docs.pytest.org/en/latest/skipping.html#id1

kain88-de · 2017-06-30T09:33:15Z

testsuite/MDAnalysisTests/core/test_atomselections.py

+        ('prop mass < 4.0 hello', universe),  # unused token
+        ('prop mass > 10. and group this', universe),  # missing group
+        ('prop mass > 10. and fullgroup this', universe),  # missing fullgroup
+    ], indirect=['universe'])


what does the indirect= do? Is that documented somewhere?

utkbansal · 2017-06-30T09:44:13Z

See: https://docs.pytest.org/en/latest/example/parametrize.html#apply-indirect-on-particular-arguments

…

On Fri, Jun 30, 2017 at 3:03 PM, Max Linke ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In testsuite/MDAnalysisTests/core/test_atomselections.py <#1443 (comment)> : > + ('resnum 7a7', universe), # rangeselection arg parsing + ('resid 1-', universe), + ('prop chicken == tasty', universe), + ('prop chicken <= 7.4', universe), + ('prop mass ^^ 12.0', universe), + ('same this as resid 1', universe), # same selection + ('same resid resname mass 5.0', universe), # same / expect + ('name H and', universe), # check all tokens used + ('naem H', universe), # unkonwn (misplet) opertaor + ('resid and name C', universe), # rangesel not finding vals + ('resnum ', universe), + ('bynum or protein', universe), + ('prop mass < 4.0 hello', universe), # unused token + ('prop mass > 10. and group this', universe), # missing group + ('prop mass > 10. and fullgroup this', universe), # missing fullgroup + ], indirect=['universe']) what does the indirect= do? Is that documented somewhere? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1443 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH9W-MVI3hel6o0uyh_Q0HgKl_qw8A9zks5sJMDcgaJpZM4OH_5a> .

richardjgowers · 2017-06-30T10:13:26Z

testsuite/MDAnalysisTests/core/test_atomselections.py

-        ]:
-            yield self.selection_fail, selstr
+
+    @pytest.mark.parametrize('selstr, universe', [


why is universe being parametrized here?

Because the function doesn't get it directly from the fixture. Have a look at the link I posted above.

Right, but why can't we just use the Universe fixture above? ie just

@parametrize('selstr', [etc etc]) def test(self, selstr, universe):

I tried that, it didn't work. I don't have a reason to why it didn't work. I'll try to look it up.

The suggestion of @richardjgowers work for me locally

richardjgowers · 2017-06-30T10:13:55Z

testsuite/MDAnalysisTests/core/test_atomselections.py


-    def selection_fail(self, selstr):
-        assert_raises(SelectionError, self.u.select_atoms,
+    def selection_fail(self, selstr, universe):


now we're not using yield (and having to provide a function to call for each test), we can collapse these functions into the test function itself

richardjgowers · 2017-06-30T10:14:33Z

testsuite/MDAnalysisTests/core/test_atomselections.py

-                       sel.format(typ=seltype))
-
-class TestICodeSelection(object):
+    @pytest.mark.parametrize('ref, sel, universe',[


same as above, why is Universe in the parametrize?

utkbansal · 2017-06-30T14:25:42Z

The drop in coverage should be fixed now, hopefully.

utkbansal · 2017-06-30T16:17:56Z

@richardjgowers @kain88-de I've got the coverage back to normal. Could you please review. Then I can do all the cleanup in one go.

richardjgowers · 2017-06-30T18:12:13Z

testsuite/MDAnalysisTests/core/test_atomselections.py


-    def _check_around(self, meth, periodic):
-        sel = Parser.parse('around 5.0 resid 1', self.u.atoms)
+    @pytest.mark.parametrize('u, meth, periodic', [


If you change the u fixture above to a staticmethod, you don't have to parametrize around it here.

ie

@staticmethod @pytest.fixture def u(): @pytest.mark.parametrize('meth, periodic', []) def test_around(self, u, meth, periodic):

will work

Oh yes, when I was trying I had a dec.skipif and that doesn't work well with static methods. That's why I got different results.

richardjgowers

needs the indirects removing first

kain88-de · 2017-07-01T05:36:11Z

testsuite/MDAnalysisTests/core/test_atomgroup.py

+        ('improper', u)
+    ], indirect=['u'])
+    def test_VEs(self, btype, u):
+        self._check_VE(btype, u)


can you directly include the _check_VE function here.

kain88-de · 2017-07-01T05:50:52Z

testsuite/MDAnalysisTests/core/test_atomselections.py

+    #             'prop mass > 10. and group this',  # missing group
+    #             'prop mass > 10. and fullgroup this',  # missing fullgroup
+    #     ]:
+    #         yield self.selection_fail, selstr


this should be removed before the final merge

kain88-de · 2017-07-01T06:06:49Z

testsuite/MDAnalysisTests/core/test_atomselections.py

-        ]:
-            yield self.selection_fail, selstr
+
+    @pytest.mark.parametrize('selstr, universe', [


The suggestion of @richardjgowers work for me locally

* Get rid of the indirect argument * Use @pytest.mark.skipif in place of @dec.skipif

richardjgowers · 2017-07-01T09:43:21Z

testsuite/MDAnalysisTests/core/test_atomgroup.py

-
-    def tearDown(self):
-        del self.u
+    @pytest.mark.skipif(parser_not_found('DCD'),'DCD parser not available. Are you using python 3?')


Can you double check that marking fixtures like this works? Ie does this mark then get transferred onto tests which use this fixture?

On my local system, I only have the minimal dependencies, all the test methods are skipped.

You run on python3? These tests will always run on Python 2

That's odd. I'm python 2.

dcd is available on python 2 as a parser. Those tests should run for you. But since we soon have a python3 DCD parser I'm not to worried if the tests are skipped but they should be run on phython2.

This should work on Python 2

@utkbansal do these tests work under python 2 now?

Yes, they work now.

richardjgowers · 2017-07-01T09:43:51Z

testsuite/MDAnalysisTests/core/test_atomselections.py

+    @staticmethod
+    @pytest.fixture()
+    def u(self):
+        # TODO: This is dummy. Needs review!


Is this still Todo, or is it Todone?

Now that we don't need to pass universe to the parametrize call, we can get rid of this completely.

utkbansal · 2017-07-02T08:55:10Z

@richardjgowers @kain88-de What else needs to be done?

kain88-de · 2017-07-02T13:48:07Z

testsuite/MDAnalysisTests/core/test_atomgroup.py

-
-    def tearDown(self):
-        del self.u
+    @pytest.mark.skipif(parser_not_found('DCD'),'DCD parser not available. Are you using python 3?')


@utkbansal do these tests work under python 2 now?

kain88-de · 2017-07-02T13:50:43Z

testsuite/MDAnalysisTests/core/test_atomselections.py

+        ('{typ} 1-5 or {typ} 7:10 or {typ} 12', '{typ} 1-5 7:10 12'),
+        ('{typ} 1 or {typ} 3 or {typ} 5:10', '{typ} 1 3 5:10'),
+    ])
+    def test_range_selections(self, seltype, ref, sel, universe):


how many tests are generated for this? Does it equal the number of tests the old double loop used in nose?

18 test cases are generated as expected.

kain88-de · 2017-07-02T16:42:55Z

@richardjgowers you should have a final look for the core module.

richardjgowers · 2017-07-02T17:02:08Z

Yep looks good

utkbansal added 5 commits June 28, 2017 19:22

Fix test_group_traj_access.py

952f626

Move core to PYTEST_LIST

5cf1c8c

Fix test_atom.py

92c878f

Fix test_atomgroup.py

e571771

Fix test_atomselections.py

4a0d977

utkbansal changed the title ~~Port core module to pytest~~ [WIP]Port core module to pytest Jun 28, 2017

utkbansal added 12 commits June 28, 2017 19:42

Fix test_groups.py

00537f7

Fix test_index_dtype.py

a14b673

Fix test_residue.py

91928cc

Fix test_residuegroup.py

1b633ec

Fix test_segment.py

50c2b74

Fix test_segmentgroup.py

4cf65e4

Fix test_topology.py

512b107

Fix test_topologyattrs.py

8d4d906

Fix test_topologyobjects.py

4383276

Fix test_universe.py

f21f2e6

Fix test_updating_atomgroup.py

dee83cc

Get rid of staticmethods

dac44c6

Get rid of yield

7260431

kain88-de reviewed Jun 30, 2017

View reviewed changes

richardjgowers reviewed Jun 30, 2017

View reviewed changes

Get rid of more yield based tests

b673a95

Get rid of another yield based test case

08d8465

richardjgowers reviewed Jun 30, 2017

View reviewed changes

richardjgowers requested changes Jun 30, 2017

View reviewed changes

kain88-de reviewed Jul 1, 2017

View reviewed changes

utkbansal added 3 commits July 1, 2017 14:41

Refactor

37c1d2e

* Get rid of the indirect argument * Use @pytest.mark.skipif in place of @dec.skipif

Remove commented out code

bee262c

Minor refactor

47311f9

richardjgowers reviewed Jul 1, 2017

View reviewed changes

Remove useless fixture

2e48c32

orbeckst added the testing label Jul 1, 2017

Import fixes

acf5f0e

kain88-de reviewed Jul 2, 2017

View reviewed changes

richardjgowers approved these changes Jul 2, 2017

View reviewed changes

utkbansal changed the title ~~[WIP]Port core module to pytest~~ Port core module to pytest Jul 2, 2017

kain88-de merged commit b97f2ab into MDAnalysis:develop Jul 2, 2017

utkbansal mentioned this pull request Jul 2, 2017

Fix coverage drop in coordinates module #1441

Merged

4 tasks

Conversation

utkbansal commented Jun 28, 2017

Changes made in this Pull Request:

PR Checklist

Uh oh!

utkbansal commented Jun 28, 2017

Uh oh!

utkbansal commented Jun 29, 2017

Uh oh!

utkbansal commented Jun 29, 2017

Uh oh!

utkbansal commented Jun 29, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

utkbansal commented Jun 29, 2017

Uh oh!

utkbansal commented Jun 29, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

richardjgowers commented Jun 29, 2017

Uh oh!

utkbansal commented Jun 29, 2017

Uh oh!

kain88-de commented Jun 29, 2017

Uh oh!

kain88-de commented Jun 29, 2017

Uh oh!

utkbansal commented Jun 29, 2017

Uh oh!

kain88-de commented Jun 29, 2017

Uh oh!

utkbansal commented Jun 29, 2017

Uh oh!

kain88-de commented Jun 29, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

utkbansal commented Jun 30, 2017 via email

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

utkbansal commented Jun 30, 2017

Uh oh!

utkbansal commented Jun 30, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

richardjgowers left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

utkbansal commented Jun 29, 2017 •

edited

Loading

utkbansal commented Jun 29, 2017 •

edited

Loading