Pytest Style: test_util.py #1494

utkbansal · 2017-07-15T21:47:29Z

Fixes #

Changes made in this Pull Request:

PR Checklist

Tests?
Docs?
CHANGELOG updated?
Issue raised/referenced?

utkbansal

@kain88-de @richardjgowers @jbarnoud @mnmelo Please have a look at this fixture I've created. I have used lambdas to save myself from writing a function that sets up the data and is then called in parametrize. Is this style okay (it works fine)?

utkbansal · 2017-07-16T12:46:48Z

testsuite/MDAnalysisTests/lib/test_util.py

+                                        map(lambda format: format[0], formats)) +
+                                        map(lambda ext: ext.lower(), map(lambda format: format[0], formats)
+    ))
+    def test_get_extention(self, ext):


This is the new function. (FYI this will be done directly on _check_get_ext - this wrapper method will not be there in the final version).

Turns out that I don't really need nested calls, I can just do

map(lambda format: format[0].upper(), formats) + map(lambda format: format[0].lower(), formats)

utkbansal · 2017-07-16T12:48:05Z

testsuite/MDAnalysisTests/lib/test_util.py

+            for extention in [format.upper(), format.lower()]:
+                file_name = 'file.{0}'.format(extention)
+                # check extention doesn't trip up get_ext or guess_format
+                yield self._check_get_ext, extention, file_name


This call is replaced by that fixture funciton.

jbarnoud

I had just a quick check. Some changes, though.

jbarnoud · 2017-07-16T15:13:28Z

package/.pylintrc

@@ -199,7 +199,6 @@ enable=abstract-class-instantiated,
       print-statement,
       xrange-builtin,
       zip-builtin-not-iterating,
-       map-builtin-not-iterating,


You should not remove that check.

That check is erroring out the build. Do you know of any alternate way to write the labdas I've written?

Just make the list somewhere in the module namespace, you keep reusing it anyway

Look bellow. It probably fails because you use + on map. In python 3, map is an iterator so you cannot use + on it. You need to convert the map output to a list to extend it.

By the way, if you use map, you should import it from six.

In general, you should never remove checks from pylint. If you understand exactly why the test fail, and you are sure about why you are doing, you may deactivate the check locally.

Instead of a map you can also use a normal list comprehension.

List comprehensions dont work. TypeError: zip argument #1 must support iteration

jbarnoud · 2017-07-16T15:21:30Z

testsuite/MDAnalysisTests/lib/test_util.py


-    def _check_compressed(self, f, fn):
+    @pytest.mark.parametrize('extention',
+        map(lambda format: format[0].upper(), formats) +


The + will break on python 3. You need to consume the iterator in a list.

turn it into a list comprehension [f[0].upper() for f in formats]. Please also try not to use keywords of the language itself.

@kain88-de List comprehensions don't work inside parametrize calls. I get a TypeError.

jbarnoud · 2017-07-16T15:26:58Z

testsuite/MDAnalysisTests/lib/test_util.py

-        assert_equal(self.obj.val3, self.obj.ref3)
-        assert_equal('val3' in self.obj._cache, True)
+        assert obj.val3 == obj.ref3
+        assert ('val3' in obj._cache) == True


'val3' in obj._cache is enough. The == True is useless.

Same above.

kain88-de · 2017-07-16T16:31:05Z

testsuite/MDAnalysisTests/lib/test_util.py

    def test_convert_aa_code_long(self):
        for resname1, strings in self.aa:
            for resname3 in strings:
-                yield check_convert_aa_3to1, resname3, resname1
+                yield self.check_convert_aa_3to1, resname3, resname1


doesn't the yield still raise a internal pytest warning? We should remove all the yields.

It is marked as TODO.

kain88-de · 2017-07-16T16:31:32Z

testsuite/MDAnalysisTests/lib/test_util.py

+    #     self.e2 = np.array([0, 1., 0])
+    #     self.e3 = np.array([0, 0, 1.])
+    #     self.a = np.array([np.cos(np.pi / 3), np.sin(np.pi / 3), 0])
+    #     self.null = np.zeros(3)


please remove these comments.

kain88-de · 2017-07-16T16:31:51Z

testsuite/MDAnalysisTests/lib/test_util.py

+class TestGeometryFunctions(object):
+    e1 = np.array([1., 0, 0])
+    e2 = np.array([0, 1., 0])
+    e3 = np.array([0, 0, 1.])


more concise would be e1, e2, e3 = np.eye(3)

kain88-de · 2017-07-16T16:32:29Z

testsuite/MDAnalysisTests/lib/test_util.py

+    # ])
+    # def test_normal(self, vec1, vec2, value):
+    #     assert mdamath.normal(vec1, vec2) == value
+    #     # add more non-trivial tests


why does this fail?

kain88-de · 2017-07-16T16:33:34Z

testsuite/MDAnalysisTests/lib/test_util.py

-        assert_equal(ret, self.obj.ref1)
+        obj._clear_caches()
+        ret = obj.val1()
+        assert ('val1' in obj._cache) == True


as @jbarnoud mentioned you don't need to do assert foo == True just using assert foo is enough

You should check the whole file for this pattern

richardjgowers · 2017-07-16T20:13:01Z

testsuite/MDAnalysisTests/lib/test_util.py


-    def _load_bonds(self):
+    def _load_bonds(self, universe):


Could we make a fixture called universe_with_bonds which is just the universe fixture and this function called? Ie put the body of this method into a fixture and use that instead?

utkbansal · 2017-07-20T13:35:13Z

I am able to get rid of lambda functions but oddly my linter complaints about references. Let's see what the Travis lint build says.

utkbansal · 2017-07-20T13:48:28Z

@richardjgowers @kain88-de Can you help me with these lint errors

testsuite/MDAnalysisTests/lib/test_util.py:596: [E1136(unsubscriptable-object), TestGuessFormat.test_get_extention] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:597: [E1136(unsubscriptable-object), TestGuessFormat.test_get_extention] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:606: [E1136(unsubscriptable-object), TestGuessFormat.test_compressed_without_compression_extention] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:607: [E1136(unsubscriptable-object), TestGuessFormat.test_compressed_without_compression_extention] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:615: [E1136(unsubscriptable-object), TestGuessFormat.test_compressed] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:616: [E1136(unsubscriptable-object), TestGuessFormat.test_compressed] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:626: [E1136(unsubscriptable-object), TestGuessFormat.test_guess_format] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:626: [E1136(unsubscriptable-object), TestGuessFormat.test_guess_format] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:634: [E1136(unsubscriptable-object), TestGuessFormat.test_guess_format_compressed] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:634: [E1136(unsubscriptable-object), TestGuessFormat.test_guess_format_compressed] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:643: [E1136(unsubscriptable-object), TestGuessFormat.test_get_parser] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:643: [E1136(unsubscriptable-object), TestGuessFormat.test_get_parser] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:643: [E1136(unsubscriptable-object), TestGuessFormat.test_get_parser] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:652: [E1136(unsubscriptable-object), TestGuessFormat.test_get_parser_compressed] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:652: [E1136(unsubscriptable-object), TestGuessFormat.test_get_parser_compressed] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:652: [E1136(unsubscriptable-object), TestGuessFormat.test_get_parser_compressed] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:662: [E1136(unsubscriptable-object), TestGuessFormat.test_get_parser_invalid] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:662: [E1136(unsubscriptable-object), TestGuessFormat.test_get_parser_invalid] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:662: [E1136(unsubscriptable-object), TestGuessFormat.test_get_parser_invalid] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:670: [E1136(unsubscriptable-object), TestGuessFormat.test_get_reader] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:670: [E1136(unsubscriptable-object), TestGuessFormat.test_get_reader] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:670: [E1136(unsubscriptable-object), TestGuessFormat.test_get_reader] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:679: [E1136(unsubscriptable-object), TestGuessFormat.test_get_reader_compressed] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:679: [E1136(unsubscriptable-object), TestGuessFormat.test_get_reader_compressed] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:679: [E1136(unsubscriptable-object), TestGuessFormat.test_get_reader_compressed] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:689: [E1136(unsubscriptable-object), TestGuessFormat.test_get_reader_invalid] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:689: [E1136(unsubscriptable-object), TestGuessFormat.test_get_reader_invalid] Value 'format' is unsubscriptable
testsuite/MDAnalysisTests/lib/test_util.py:689: [E1136(unsubscriptable-object), TestGuessFormat.test_get_reader_invalid] Value 'format' is unsubscriptable

My IDE also complaints the same, but it is subscriptable! The tests pass on my system.

richardjgowers · 2017-07-20T13:52:55Z

@utkbansal it's because format is a builtin function, and the linter doesn't understand that you've shadowed it. Change it to something else and it should be fine

kain88-de · 2017-07-20T14:08:38Z

testsuite/MDAnalysisTests/lib/test_util.py

            ('TRZ', None, mda.coordinates.TRZ.TRZReader),
        ]
    # list of possible compressed extensions
    # include no extension too!
    compressed_extensions = ['.bz2', '.gz']

-    def _check_get_ext(self, f, fn):
+    @pytest.mark.parametrize('extention', [format[0].upper() for format in formats] +


what happens if you choose another name. A static analyzer might confuse this with the built-in format.

@kain88-de @richardjgowers Let's say I rename format to foo. Then the linter complaints Unresolved reference The tests still work though.

I don't believe that. Please give it a try and let travis run on it.

utkbansal · 2017-07-20T16:01:34Z

@kain88-de @richardjgowers The build passed. I still have lint errors in my IDE though. Is there anything else that needs to be done here?

richardjgowers · 2017-07-20T16:16:11Z

testsuite/MDAnalysisTests/lib/test_util.py

-
-def check_convert_aa_1to3(resname1, resname3_canonical):
-    assert_equal(util.convert_aa_code(resname1), resname3_canonical)
+def test_convert_aa_code_long_data():


Is this getting also ran as a test because of the name?

Could this get rewritten as a generator?

Can think of a list comprehension to replace that.

jbarnoud

You need to rebase against develop and to fix the order of the imports.

jbarnoud · 2017-07-22T12:35:43Z

testsuite/MDAnalysisTests/lib/test_util.py

+    data = []
+    for resname1, strings in aa:
+        for resname3 in strings:
+            data.append((resname3, resname1))


This could be a generator. Would not change much, though.

@kain88-de @jbarnoud I need help converting this to a list comprehension.

The point is not to make it a list comprehension, but a generator. To do that, replace this line by yield (resname3, resname1), remove the data list, and remove the return. But as I said, it does not change much, I like the generator better but the list is fine too.

jbarnoud · 2017-07-22T12:37:39Z

testsuite/MDAnalysisTests/lib/test_util.py

 import MDAnalysis.lib.mdamath as mdamath
-from MDAnalysis.lib.util import cached
+import MDAnalysis.lib.util as util


The imports are disordered.

jbarnoud · 2017-07-22T12:42:41Z

testsuite/MDAnalysisTests/lib/test_util.py


    def test_VE_1(self):
-        assert_raises(ValueError, util.convert_aa_code, 'XYZXYZ')
+        with pytest.raises(ValueError):


test_VE_1 and test_VE_2 test the same thing, it could be a parametrized test. It could also be renamed to test_ValueError or something.

kain88-de · 2017-07-23T06:37:08Z

testsuite/MDAnalysisTests/lib/test_util.py

+    ])
+    def test_string(self, name, ext, keep, actual_name):
+        file_name = util.filename(name, ext, keep)
+        assert file_name, actual_name #TODO: why is a comma here?


Please fix the todo

kain88-de · 2017-07-26T20:50:43Z

@utkbansal can you finish this up.

jbarnoud · 2017-07-27T14:53:48Z

testsuite/MDAnalysisTests/lib/test_util.py

@@ -605,103 +591,138 @@ class TestGuessFormat(object):
        ('XYZ', mda.topology.XYZParser.XYZParser, mda.coordinates.XYZ.XYZReader),
    ]
    if six.PY2:
-        # DCD, LAMMPS, and TRZ are not supported on Python 3 yet
+        # TRZ is not supported on Python 3 yet


Do we still need that?

Everything is supported in py3, this should get rebased onto develop

So should I remove the check?

richardjgowers · 2017-07-27T14:57:49Z

testsuite/MDAnalysisTests/lib/test_util.py

@@ -817,32 +819,38 @@ def _check_multiframe_fails(fmt):
    ]
    if six.PY2:


this if can get removed too

jbarnoud · 2017-07-27T16:21:04Z

Yes, there should not be special cases for python 3 anymore.

…

On 07/27/2017 06:18 PM, Utkarsh Bansal wrote: ***@***.**** commented on this pull request. ------------------------------------------------------------------------ In testsuite/MDAnalysisTests/lib/test_util.py <#1494 (comment)>: > @@ -605,103 +591,138 @@ class TestGuessFormat(object): ('XYZ', mda.topology.XYZParser.XYZParser, mda.coordinates.XYZ.XYZReader), ] if six.PY2: - # DCD, LAMMPS, and TRZ are not supported on Python 3 yet + # TRZ is not supported on Python 3 yet So should I remove the check? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1494 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUWup3Kl_RvnCxOUE2cyEyn_usliiljks5sSLhngaJpZM4OZInh>.

kain88-de · 2017-07-27T17:19:26Z

testsuite/MDAnalysisTests/lib/test_util.py

    for resname1, strings in aa:
        for resname3 in strings:
-            data.append((resname3, resname1))
+            yield (resname3, resname1)


Wasn't the goal to remove yield?

No, this isn't a yield based test. @jbarnoud recommended this #1494 (comment)

utkbansal · 2017-07-27T17:58:22Z

@kain88-de @jbarnoud Better now?

jbarnoud

Sorry, a few more comments. In addition, all the tests that have a exception name abbreviated in their name could be rename to avoid the abbreviation. There are a few tests named test_*_VE or test_*_TE.

jbarnoud · 2017-07-27T19:52:17Z

testsuite/MDAnalysisTests/lib/test_util.py

        # Does ``get_writer_for`` fails with non string filename, no format
+        with pytest.raises(ValueError):
+            mda.coordinates.core.get_writer_for(filename = StringIO(), format = None)


There should not be spaces around the =.

jbarnoud · 2017-07-27T19:52:31Z

testsuite/MDAnalysisTests/lib/test_util.py

        # does ``get_writer_for`` fail with invalid format and multiframe not None
+        with pytest.raises(TypeError):
+            mda.coordinates.core.get_writer_for(filename = "fail_me", format = 'UNK', multiframe = True)


jbarnoud · 2017-07-27T19:52:40Z

testsuite/MDAnalysisTests/lib/test_util.py

        # does ``get_writer_for`` fail with invalid format and multiframe not None
+        with pytest.raises(TypeError):
+            mda.coordinates.core.get_writer_for(filename = "fail_me", format = 'UNK', multiframe = True)
+            mda.coordinates.core.get_writer_for(filename = "fail_me", format = 'UNK', multiframe = False)


jbarnoud · 2017-07-27T19:52:47Z

testsuite/MDAnalysisTests/lib/test_util.py

-                      mda.coordinates.core.get_writer_for,
-                      'this', format=fmt, multiframe=True)
+        with pytest.raises(ValueError):
+            mda.coordinates.core.get_writer_for(filename = 'this.gro', multiframe = 'sandwich')


jbarnoud · 2017-07-27T19:53:51Z

testsuite/MDAnalysisTests/lib/test_util.py

-    def test_setattr(self):
-        self.ns.this = 42
+        with pytest.raises(AttributeError):
+            deller()


Does not need to be a nested function.

jbarnoud · 2017-07-27T19:58:51Z

testsuite/MDAnalysisTests/lib/test_util.py

-        assert_equal(ret['delta'], 0.4)
-        assert_almost_equal(ret['min'], 3.9)
-        assert_almost_equal(ret['max'], 5.1)
+        with pytest.raises(ValueError):


Rename test_VE to test_ValueError.

kain88-de · 2017-07-29T07:45:25Z

@utkbansal this isn't fully resolved yet

utkbansal · 2017-07-29T16:02:15Z

@kain88-de @jbarnoud Fixed the whitespace and naming issues.

utkbansal changed the title ~~Pytest Style: test_util.py~~ [WIP]Pytest Style: test_util.py Jul 15, 2017

utkbansal commented Jul 16, 2017

View reviewed changes

jbarnoud requested changes Jul 16, 2017

View reviewed changes

kain88-de requested changes Jul 16, 2017

View reviewed changes

richardjgowers reviewed Jul 16, 2017

View reviewed changes

kain88-de reviewed Jul 20, 2017

View reviewed changes

utkbansal changed the title ~~[WIP]Pytest Style: test_util.py~~ Pytest Style: test_util.py Jul 20, 2017

richardjgowers reviewed Jul 20, 2017

View reviewed changes

jbarnoud requested changes Jul 22, 2017

View reviewed changes

kain88-de reviewed Jul 23, 2017

View reviewed changes

utkbansal added 15 commits July 27, 2017 19:54

Fixes

10fff22

Fix TestMakeWhole

a74065b

Fix TestCachedDecorator

32fb94a

Fix TestConvFloat

cd36453

WIP TestGuessFormat

a8aabed

WIP TestGuessFormat

b752ed2

WIP3 TestGuessFormat

69c3d18

WIP4 TestGuessFormat

386bd18

Fix TestGuessFormat

80127f8

Supress pylint warning

669e4b8

Fix TestGetWriterFor

4e90566

Fix TestNamespace

4b5da75

Fix TestTruncateInteger

8063e0e

Cleanup

9c58369

Do _load_bonds in fixture

92d6b43

utkbansal added 6 commits July 27, 2017 19:55

Move body of function to fixture

909afeb

Convert map to list comprehensions

fd61146

Remove all calls to lambda

da55117

Rename format to format_tuple

d584579

Cleanup

08c3a43

Cleanup

eebbb46

utkbansal force-pushed the util branch from 89a1168 to eebbb46 Compare July 27, 2017 14:38

jbarnoud reviewed Jul 27, 2017

View reviewed changes

richardjgowers reviewed Jul 27, 2017

View reviewed changes

Remove python3 checks

6544754

kain88-de reviewed Jul 27, 2017

View reviewed changes

jbarnoud requested changes Jul 27, 2017

View reviewed changes

Whitespace and name fixes

c601ee8

jbarnoud approved these changes Jul 29, 2017

View reviewed changes

kain88-de approved these changes Jul 29, 2017

View reviewed changes

kain88-de merged commit a1e49ba into MDAnalysis:develop Jul 29, 2017

		@@ -817,32 +819,38 @@ def _check_multiframe_fails(fmt):
		]
		if six.PY2:

Pytest Style: test_util.py #1494

Pytest Style: test_util.py #1494

Conversation

utkbansal commented Jul 15, 2017

Changes made in this Pull Request:

PR Checklist

utkbansal left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbarnoud left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

utkbansal Jul 16, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

utkbansal commented Jul 20, 2017

utkbansal commented Jul 20, 2017

richardjgowers commented Jul 20, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

utkbansal commented Jul 20, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbarnoud left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kain88-de commented Jul 26, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbarnoud commented Jul 27, 2017 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

utkbansal commented Jul 27, 2017

jbarnoud left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kain88-de commented Jul 29, 2017

utkbansal commented Jul 29, 2017

utkbansal Jul 16, 2017 •

edited

Loading