Pytest Style coordinates/test_xdr.py #1655

utkbansal · 2017-08-26T09:05:59Z

@richardjgowers @kain88-de @jbarnoud The tests in _GromacsReader_offsets are failing.
My best guess is that it is because of the line shutil.copy(self.filename, str(tmpdir))(is in the setUp method). What have I done wrong?

kain88-de · 2017-08-26T17:03:46Z

Can you run those tests locally?

utkbansal · 2017-08-26T17:09:08Z

@kain88-de No, as I said they fail after I did my changes.

kain88-de · 2017-08-26T17:46:09Z

Can you isolate the change responsible?

orbeckst · 2017-08-28T06:16:18Z

@utkbansal could you please also remove importorskip('netCDF4') (if it is still there in test_xdr.py)? See #1650 (comment)

Thanks!

orbeckst

See comments on the offsets.

Please also remove the importorskip('netCDF4').

orbeckst · 2017-08-28T06:39:26Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

    """Issue 117: Cannot write XTC or TRR from AMBER NCDF"""
    ext = None
    prec = 5

-    def setUp(self):
+    @pytest.fixture()
+    def universe(self):
        pytest.importorskip('netCDF4')


Please remove pytest.importorskip('netCDF4') - not needed anymore since #503. Thanks.

orbeckst · 2017-08-28T06:39:26Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

+    @pytest.fixture()
+    def ts(self, trajectory):
+        return trajectory.ts
+

    def setUp(self):


Don't you have to delete setUp, now that you have the fixtures?

orbeckst · 2017-08-28T06:39:26Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

+    @pytest.fixture()
+    def ts(self, trajectory):
+        return trajectory.ts
+

    def setUp(self):
        # since offsets are automatically generated in the same directory


Put this 2-line comment before the new fixtures – it is important in order to understand why we even bother to copy trajectories around.

orbeckst · 2017-08-28T06:39:26Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

+    def trajectory(self, traj):
+        return self._reader(traj)
+
+    @pytest.fixture()


The ts fixture seems overkill; I don't think it is wrong but looks overly complicated. I would rewrite the tests to use trajectory.ts instead of ts.

orbeckst · 2017-08-28T06:39:26Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

@@ -751,6 +720,21 @@ class _GromacsReader_offsets(TestCase):
    ref_volume = 362270.0
    ref_offsets = None
    _reader = None
+    prec = 3
+
+    @pytest.fixture()


I think, this must be a function/method level-fixture. It cannot be re-used for the whole class. I don't know if this is its default scope. — @kain88-de please correct me if I am wrong here.

The reason is that MDAnalysis creates a hidden file when it reads the trajectory the first time (the so-called offsets file). This hidden file is stored in the same directory as the trajectory file. The tests here are supposed to test what happens when this hidden file does not exist yet.

@orbeckst by default things are function scoped

kain88-de · 2017-09-05T17:39:19Z

@orbeckst can you have another look. The tests should be fixed now.

orbeckst

@kain88-de Thanks for fixing up everything. Sorry, I have a few more change requests, primarily just using context managers for the writers when possible. Please ping me when you want me to look at it again.

orbeckst · 2017-09-05T18:46:11Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py


+    @pytest.fixture(scope='class')


Is it safe to have only one universe per class, given that various tests rewind or use next?

Or does parallel pytest create one class-scoped fixture per process so that there is no parallel collision?

I assume the latter... just asking for clarification.

There is an experimental grouping option so that every test in a class runs on the same process. This isn't guaranteed currently. But I haven't found any errors so far. I'm also not sure when fixtures are evaluated. This is because all tests are scanned by all workers before xdist runs all tests. This likely means that each process gets it's own copy and tmp directory.

orbeckst · 2017-09-05T18:46:11Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

        # coordinates in the base unit (needed for True)
        ca_Angstrom = ca_nm * 10.0
-        U = self.universe
+        U = universe


This aliasing seems superfluous, and further down you use universe instead of U. I would just use universe everywhere.

orbeckst · 2017-09-05T18:46:11Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        assert_equal(self.universe.atoms.n_atoms, W.n_atoms)
+    def test_get_Writer(self, universe, tmpdir):
+        ext = os.path.splitext(self.filename)[1]
+        outfile = str(tmpdir.join('/xdr-reader-test' + ext))


Do you need the initial / for tmpdir.join() or does it insert the separator by itself?

orbeckst · 2017-09-05T18:46:11Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

+    def test_get_Writer(self, universe, tmpdir):
+        ext = os.path.splitext(self.filename)[1]
+        outfile = str(tmpdir.join('/xdr-reader-test' + ext))
+        W = universe.trajectory.Writer(outfile)


Please use

with universe.trajectory.Writer(outfile) as W: ...

instead of the explicit W.close(). The context manager is safer and also the "best practice".

I was simply to lazy to change all the occurrences.

orbeckst · 2017-09-05T18:46:11Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

+        universe = mda.Universe(GRO, self.filename, convert_units=True)
+        ext = os.path.splitext(self.filename)[1]
+        outfile = str(tmpdir.join('/xdr-reader-test' + ext))
+        W = universe.trajectory.Writer(outfile)


Please use context manager.

orbeckst · 2017-09-05T18:46:11Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        for ts in self.universe.trajectory:
+    def test_velocities(self, universe, Writer, outfile):
+        t = universe.trajectory
+        W = Writer(outfile, t.n_atoms, dt=t.dt)


Context manager?

orbeckst · 2017-09-05T18:46:11Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        W = self.Writer(self.outfile, t.n_atoms, dt=t.dt)
-        for ts in self.universe.trajectory:
+    def test_velocities(self, universe, Writer, outfile):
+        t = universe.trajectory


Gratuitous aliasing ;-) – but that's a matter of taste.

orbeckst · 2017-09-05T18:46:11Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        W = self.Writer(self.outfile, t.n_atoms, dt=t.dt)
-        for ts in self.universe.trajectory:
+        t = universe.trajectory
+        W = Writer(outfile, t.n_atoms, dt=t.dt)


Context manager?

orbeckst · 2017-09-05T18:46:11Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        t = self.universe.trajectory
-        W = self.Writer(self.outfile, t.n_atoms, dt=t.dt)
-        for ts in self.universe.trajectory:
+        t = universe.trajectory


Maybe just write it out in the next line...

orbeckst · 2017-09-05T18:46:11Z

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        self.Writer.close()
+        universe.trajectory
+        outfile = str(tmpdir.join('/xdr-writer-issue117' + self.ext))
+        Writer = mda.Writer(outfile, n_atoms=universe.atoms.n_atoms)


Context manager?

kain88-de · 2017-09-05T20:20:50Z

@orbeckst ping.

orbeckst · 2017-09-05T20:52:24Z

Just waiting for travis.

I will squash it all together. I think this will credit the commit to @kain88-de because the first commit was already committed with him. Is this ok, @utkbansal and @kain88-de ? Alternatively, rewrite the history yourself into two separate commits. But I would want to avoid multiple minor commits.

- fix failing tests - remove setUp method - remove array asserts - clean up - restore old comment - style fixes - use best practices with context manager - class scope for fixture

kain88-de · 2017-09-05T20:54:34Z

@orbeckst please rebase or merge normally. Do not squash.

kain88-de · 2017-09-05T20:55:54Z

btw travis builds can take a good while now. The osx builds take forever to start.

orbeckst · 2017-09-05T20:57:26Z

Then can you please rewrite the history and squash all your commits yourself? I will then merge the two commits, one from @utkbansal, the other from you. But I don't see a point in having all these little changes in a single file split into multiple commits – or have you got specific reasons for keeping your commits separate?

kain88-de · 2017-09-05T21:07:54Z

@orbeckst done

orbeckst · 2017-09-05T21:09:09Z

Thanks!! ... just waiting for the green checkmark from Travis and will then merge.

orbeckst · 2017-09-05T23:47:01Z

The macOS tests have not been scheduled to run yet on Travis but everything else passes. Therefore, I am going to merge.

orbeckst mentioned this pull request Aug 28, 2017

[Review Needed]Pytest Style coordinates/test_timestep_api.py #1650

Merged

0 of 4 tasks complete

orbeckst reviewed Aug 28, 2017

View changes

Pytest Style coordinates/test_xdr.py

e377b79

kain88-de force-pushed the utkbansal:coord-xdr branch from bbf0050 to 998a3b7 Sep 5, 2017

orbeckst requested changes Sep 5, 2017

View changes

orbeckst changed the title ~~[Review Needed]Pytest Style coordinates/test_xdr.py~~ Pytest Style coordinates/test_xdr.py Sep 5, 2017

orbeckst approved these changes Sep 5, 2017

View changes

finish pytest transition

Loading status checks…

ac47b78

- fix failing tests - remove setUp method - remove array asserts - clean up - restore old comment - style fixes - use best practices with context manager - class scope for fixture

kain88-de force-pushed the utkbansal:coord-xdr branch from 8142a6a to ac47b78 Sep 5, 2017

orbeckst merged commit 5e2d0ba into MDAnalysis:develop Sep 5, 2017
0 of 2 checks passed

0 of 2 checks passed

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details

coverage/coveralls Coverage pending from Coveralls.io
Details

MDAnalysis / mdanalysis

Pytest Style coordinates/test_xdr.py #1655

Pytest Style coordinates/test_xdr.py #1655

utkbansal commented Aug 26, 2017 •

edited

kain88-de commented Aug 26, 2017

utkbansal commented Aug 26, 2017

kain88-de commented Aug 26, 2017

orbeckst commented Aug 28, 2017

orbeckst left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

kain88-de commented Sep 5, 2017

orbeckst left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

kain88-de commented Sep 5, 2017

orbeckst commented Sep 5, 2017 •

edited

kain88-de commented Sep 5, 2017

kain88-de commented Sep 5, 2017

orbeckst commented Sep 5, 2017

kain88-de commented Sep 5, 2017

orbeckst commented Sep 5, 2017

orbeckst commented Sep 5, 2017

MDAnalysis / mdanalysis

Pytest Style coordinates/test_xdr.py #1655

Pytest Style coordinates/test_xdr.py #1655

Conversation

utkbansal commented Aug 26, 2017 • edited

kain88-de commented Aug 26, 2017

utkbansal commented Aug 26, 2017

kain88-de commented Aug 26, 2017

orbeckst commented Aug 28, 2017

orbeckst left a comment

This comment has been minimized.

orbeckst Aug 28, 2017 Member

This comment has been minimized.

orbeckst Aug 28, 2017 Member

This comment has been minimized.

orbeckst Aug 28, 2017 Member

This comment has been minimized.

orbeckst Aug 28, 2017 Member

This comment has been minimized.

orbeckst Aug 28, 2017 Member

This comment has been minimized.

richardjgowers Aug 28, 2017 Member

kain88-de commented Sep 5, 2017

orbeckst left a comment

This comment has been minimized.

orbeckst Sep 5, 2017 Member

This comment has been minimized.

kain88-de Sep 5, 2017 Member

This comment has been minimized.

orbeckst Sep 5, 2017 Member

This comment has been minimized.

orbeckst Sep 5, 2017 Member

This comment has been minimized.

orbeckst Sep 5, 2017 Member

This comment has been minimized.

kain88-de Sep 5, 2017 Member

This comment has been minimized.

orbeckst Sep 5, 2017 Member

This comment has been minimized.

orbeckst Sep 5, 2017 Member

This comment has been minimized.

orbeckst Sep 5, 2017 Member

This comment has been minimized.

orbeckst Sep 5, 2017 Member

This comment has been minimized.

orbeckst Sep 5, 2017 Member

This comment has been minimized.

orbeckst Sep 5, 2017 Member

kain88-de commented Sep 5, 2017

orbeckst commented Sep 5, 2017 • edited

kain88-de commented Sep 5, 2017

kain88-de commented Sep 5, 2017

orbeckst commented Sep 5, 2017

kain88-de commented Sep 5, 2017

orbeckst commented Sep 5, 2017

orbeckst commented Sep 5, 2017

utkbansal commented Aug 26, 2017 •

edited

orbeckst Aug 28, 2017
Member

orbeckst Aug 28, 2017
Member

orbeckst Aug 28, 2017
Member

orbeckst Aug 28, 2017
Member

orbeckst Aug 28, 2017
Member

richardjgowers Aug 28, 2017
Member

orbeckst Sep 5, 2017
Member

kain88-de Sep 5, 2017
Member

orbeckst Sep 5, 2017
Member

orbeckst Sep 5, 2017
Member

orbeckst Sep 5, 2017
Member

kain88-de Sep 5, 2017
Member

orbeckst Sep 5, 2017
Member

orbeckst Sep 5, 2017
Member

orbeckst Sep 5, 2017
Member

orbeckst Sep 5, 2017
Member

orbeckst Sep 5, 2017
Member

orbeckst Sep 5, 2017
Member

orbeckst commented Sep 5, 2017 •

edited