Pytest Style coordinates/test_xdr.py #1655

utkbansal · Aug 26, 2017

@richardjgowers @kain88-de @jbarnoud The tests in _GromacsReader_offsets are failing.
My best guess is that it is because of the line shutil.copy(self.filename, str(tmpdir))(is in the setUp method). What have I done wrong?

kain88-de · Aug 26, 2017

Can you run those tests locally?

utkbansal · Aug 26, 2017

@kain88-de No, as I said they fail after I did my changes.

kain88-de · Aug 26, 2017

Can you isolate the change responsible?

orbeckst · Aug 28, 2017

@utkbansal could you please also remove importorskip('netCDF4') (if it is still there in test_xdr.py)? See #1650 (comment)

Thanks!

orbeckst · Aug 28, 2017

orbeckst reviewed Aug 28, 2017

View changes

See comments on the offsets.

Please also remove the importorskip('netCDF4').

orbeckst · Aug 28, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

    """Issue 117: Cannot write XTC or TRR from AMBER NCDF"""
    ext = None
    prec = 5

-    def setUp(self):
+    @pytest.fixture()
+    def universe(self):
        pytest.importorskip('netCDF4')


Please remove pytest.importorskip('netCDF4') - not needed anymore since #503. Thanks.

orbeckst · Aug 28, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

+    @pytest.fixture()
+    def ts(self, trajectory):
+        return trajectory.ts
+    

    def setUp(self):


Don't you have to delete setUp, now that you have the fixtures?

orbeckst · Aug 28, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

+    @pytest.fixture()
+    def ts(self, trajectory):
+        return trajectory.ts
+    

    def setUp(self):
        # since offsets are automatically generated in the same directory


Put this 2-line comment before the new fixtures – it is important in order to understand why we even bother to copy trajectories around.

orbeckst · Aug 28, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

+    def trajectory(self, traj):
+        return self._reader(traj)
+
+    @pytest.fixture()


The ts fixture seems overkill; I don't think it is wrong but looks overly complicated. I would rewrite the tests to use trajectory.ts instead of ts.

orbeckst · Aug 28, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

@@ -751,6 +720,21 @@ class _GromacsReader_offsets(TestCase):
    ref_volume = 362270.0
    ref_offsets = None
    _reader = None
+    prec = 3
+    
+    @pytest.fixture()


I think, this must be a function/method level-fixture. It cannot be re-used for the whole class. I don't know if this is its default scope. — @kain88-de please correct me if I am wrong here.

The reason is that MDAnalysis creates a hidden file when it reads the trajectory the first time (the so-called offsets file). This hidden file is stored in the same directory as the trajectory file. The tests here are supposed to test what happens when this hidden file does not exist yet.

@orbeckst by default things are function scoped

kain88-de · Sep 5, 2017

@orbeckst can you have another look. The tests should be fixed now.

orbeckst · Sep 5, 2017

orbeckst requested changes Sep 5, 2017

View changes

@kain88-de Thanks for fixing up everything. Sorry, I have a few more change requests, primarily just using context managers for the writers when possible. Please ping me when you want me to look at it again.

orbeckst · Sep 5, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py


+    @pytest.fixture(scope='class')


Is it safe to have only one universe per class, given that various tests rewind or use next?

Or does parallel pytest create one class-scoped fixture per process so that there is no parallel collision?

I assume the latter... just asking for clarification.

There is an experimental grouping option so that every test in a class runs on the same process. This isn't guaranteed currently. But I haven't found any errors so far. I'm also not sure when fixtures are evaluated. This is because all tests are scanned by all workers before xdist runs all tests. This likely means that each process gets it's own copy and tmp directory.

orbeckst · Sep 5, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

        # coordinates in the base unit (needed for True)
        ca_Angstrom = ca_nm * 10.0
-        U = self.universe
+        U = universe


This aliasing seems superfluous, and further down you use universe instead of U. I would just use universe everywhere.

orbeckst · Sep 5, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        assert_equal(self.universe.atoms.n_atoms, W.n_atoms)
+    def test_get_Writer(self, universe, tmpdir):
+        ext = os.path.splitext(self.filename)[1]
+        outfile = str(tmpdir.join('/xdr-reader-test' + ext))


Do you need the initial / for tmpdir.join() or does it insert the separator by itself?

orbeckst · Sep 5, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

+    def test_get_Writer(self, universe, tmpdir):
+        ext = os.path.splitext(self.filename)[1]
+        outfile = str(tmpdir.join('/xdr-reader-test' + ext))
+        W = universe.trajectory.Writer(outfile)


Please use

with universe.trajectory.Writer(outfile) as W: ...

instead of the explicit W.close(). The context manager is safer and also the "best practice".

I was simply to lazy to change all the occurrences.

orbeckst · Sep 5, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

+        universe = mda.Universe(GRO, self.filename, convert_units=True)
+        ext = os.path.splitext(self.filename)[1]
+        outfile = str(tmpdir.join('/xdr-reader-test' + ext))
+        W = universe.trajectory.Writer(outfile)


Please use context manager.

orbeckst · Sep 5, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        for ts in self.universe.trajectory:
+    def test_velocities(self, universe, Writer, outfile):
+        t = universe.trajectory
+        W = Writer(outfile, t.n_atoms, dt=t.dt)


Context manager?

orbeckst · Sep 5, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        W = self.Writer(self.outfile, t.n_atoms, dt=t.dt)
-        for ts in self.universe.trajectory:
+    def test_velocities(self, universe, Writer, outfile):
+        t = universe.trajectory


Gratuitous aliasing ;-) – but that's a matter of taste.

orbeckst · Sep 5, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        W = self.Writer(self.outfile, t.n_atoms, dt=t.dt)
-        for ts in self.universe.trajectory:
+        t = universe.trajectory
+        W = Writer(outfile, t.n_atoms, dt=t.dt)


Context manager?

orbeckst · Sep 5, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        t = self.universe.trajectory
-        W = self.Writer(self.outfile, t.n_atoms, dt=t.dt)
-        for ts in self.universe.trajectory:
+        t = universe.trajectory


Maybe just write it out in the next line...

orbeckst · Sep 5, 2017

testsuite/MDAnalysisTests/coordinates/test_xdr.py

-        self.Writer.close()
+        universe.trajectory
+        outfile = str(tmpdir.join('/xdr-writer-issue117' + self.ext))
+        Writer = mda.Writer(outfile, n_atoms=universe.atoms.n_atoms)


Context manager?

kain88-de · Sep 5, 2017

@orbeckst ping.

orbeckst · Sep 5, 2017

orbeckst approved these changes Sep 5, 2017

View changes

orbeckst · Sep 5, 2017

Just waiting for travis.

I will squash it all together. I think this will credit the commit to @kain88-de because the first commit was already committed with him. Is this ok, @utkbansal and @kain88-de ? Alternatively, rewrite the history yourself into two separate commits. But I would want to avoid multiple minor commits.

kain88-de · Sep 5, 2017

@orbeckst please rebase or merge normally. Do not squash.

kain88-de · Sep 5, 2017

btw travis builds can take a good while now. The osx builds take forever to start.

orbeckst · Sep 5, 2017

Then can you please rewrite the history and squash all your commits yourself? I will then merge the two commits, one from @utkbansal, the other from you. But I don't see a point in having all these little changes in a single file split into multiple commits – or have you got specific reasons for keeping your commits separate?

kain88-de · Sep 5, 2017

@orbeckst done

orbeckst · Sep 5, 2017

Thanks!! ... just waiting for the green checkmark from Travis and will then merge.

orbeckst · Sep 5, 2017

The macOS tests have not been scheduled to run yet on Travis but everything else passes. Therefore, I am going to merge.

orbeckst referenced this pull request Aug 28, 2017
Merged
[Review Needed]Pytest Style coordinates/test_timestep_api.py #1650
0 of 4 tasks complete

Pytest Style coordinates/test_xdr.py

e377b79

orbeckst changed the title from [Review Needed]Pytest Style coordinates/test_xdr.py to Pytest Style coordinates/test_xdr.py Sep 5, 2017

finish pytest transition
- fix failing tests - remove setUp method - remove array asserts - clean up - restore old comment - style fixes - use best practices with context manager - class scope for fixture

Loading status checks…

ac47b78

orbeckst merged commit 5e2d0ba into MDAnalysis:develop Sep 5, 2017
0 of 2 checks passed

0 of 2 checks passed

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details

coverage/coveralls Coverage pending from Coveralls.io
Details

MDAnalysis/mdanalysis

Join GitHub today

Pytest Style coordinates/test_xdr.py #1655

Conversation

utkbansal commented Aug 26, 2017 • edited

This comment has been minimized.

kain88-de commented Aug 26, 2017

This comment has been minimized.

utkbansal commented Aug 26, 2017

This comment has been minimized.

kain88-de commented Aug 26, 2017

orbeckst referenced this pull request Aug 28, 2017

This comment has been minimized.

orbeckst commented Aug 28, 2017

orbeckst reviewed Aug 28, 2017 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

kain88-de commented Sep 5, 2017

orbeckst requested changes Sep 5, 2017 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

orbeckst changed the title from [Review Needed]Pytest Style coordinates/test_xdr.py to Pytest Style coordinates/test_xdr.py Sep 5, 2017

This comment has been minimized.

kain88-de commented Sep 5, 2017

orbeckst approved these changes Sep 5, 2017 View changes

This comment has been minimized.

orbeckst commented Sep 5, 2017 • edited

This comment has been minimized.

kain88-de commented Sep 5, 2017

This comment has been minimized.

kain88-de commented Sep 5, 2017

This comment has been minimized.

orbeckst commented Sep 5, 2017

This comment has been minimized.

kain88-de commented Sep 5, 2017

This comment has been minimized.

orbeckst commented Sep 5, 2017

This comment has been minimized.

orbeckst commented Sep 5, 2017

Hide details View details orbeckst merged commit 5e2d0ba into MDAnalysis:develop Sep 5, 2017 0 of 2 checks passed

0 of 2 checks passed

utkbansal commented Aug 26, 2017 •

edited

orbeckst reviewed Aug 28, 2017

View changes

orbeckst requested changes Sep 5, 2017

View changes

orbeckst approved these changes Sep 5, 2017

View changes

orbeckst commented Sep 5, 2017 •

edited

orbeckst merged commit `5e2d0ba` into MDAnalysis:develop Sep 5, 2017
0 of 2 checks passed