Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iterating with TRR reader ignores IOError from incomplete/damaged frame #2267

Open
ianmkenney opened this issue May 24, 2019 · 4 comments

Comments

@ianmkenney
Copy link

commented May 24, 2019

Expected behavior

I have a TRR "000001-00043359.trr" whose last frame is damaged (as shown by gmx check -f 000001-00043359.trr.

For a TRR trajectory with a damaged frame, iteration

for i in U1.trajectory:
     pass

should fail with an error such as IOError.

It should be the same error that comes up when I access the damaged frame directly:

In [34]: U1 = mda.Universe("input_structure.dms", "000001-00043359.trr")
~/.virtualenvs/napaanton2/local/lib/python2.7/site-packages/MDAnalysis/coordinates/XDR.py:195: UserWarning: Reload offsets from trajectory
 ctime or size or n_atoms did not match
  warnings.warn("Reload offsets from trajectory\n "

In [35]: U1.trajectory[-1]
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-35-34a3a18136e5> in <module>()
----> 1 U1.trajectory[-1]

~/.virtualenvs/napaanton2/local/lib/python2.7/site-packages/MDAnalysis/coordinates/base.pyc in __getitem__(self, frame)
   1536         if isinstance(frame, numbers.Integral):
   1537             frame = self._apply_limits(frame)
-> 1538             return self._read_frame_with_aux(frame)
   1539         elif isinstance(frame, (list, np.ndarray)):
   1540             if len(frame) != 0 and isinstance(frame[0], (bool, np.bool_)):

~/.virtualenvs/napaanton2/local/lib/python2.7/site-packages/MDAnalysis/coordinates/base.pyc in _read_frame_with_aux(self, frame)
   1568     def _read_frame_with_aux(self, frame):
   1569         """Move to *frame*, updating ts with trajectory, transformations and auxiliary data."""
-> 1570         ts = self._read_frame(frame)  # pylint: disable=assignment-from-no-return
   1571         for aux in self.aux_list:
   1572             ts = self._auxs[aux].update_ts(ts)

~/.virtualenvs/napaanton2/local/lib/python2.7/site-packages/MDAnalysis/coordinates/XDR.pyc in _read_frame(self, i)
    240             self._read_offsets(store=True)
    241             self._xdr.seek(i)
--> 242             timestep = self._read_next_timestep()
    243         return timestep
    244 

~/.virtualenvs/napaanton2/local/lib/python2.7/site-packages/MDAnalysis/coordinates/XDR.pyc in _read_next_timestep(self, ts)
    249         if ts is None:
    250             ts = self.ts
--> 251         frame = self._xdr.read()
    252         self._frame += 1
    253         self._frame_to_ts(frame, ts)

MDAnalysis/lib/formats/libmdaxdr.pyx in MDAnalysis.lib.formats.libmdaxdr.TRRFile.read()

IOError: TRR read error = float

Actual behavior

Iterating seems to work just fine:

In [36]: for i in U1.trajectory:
    ...:     pass
    ...: 

In [37]: 

Actually, the last (damaged) frame is ignored and i only iterates up to the pen-ultimate frame.

EDITS: @orbeckst added a few more details from conversation with @ianmkenney

@orbeckst orbeckst changed the title iterating with TRR reader stops prematurely and ignores IOError iterating with TRR reader ignores IOError from incomplete/damaged frame May 24, 2019

@orbeckst

This comment has been minimized.

Copy link
Member

commented May 24, 2019

@ianmkenney I clarified the description a bit, after our discussion.

@orbeckst

This comment has been minimized.

Copy link
Member

commented May 24, 2019

Note that this was found because the ChainReader throws an error while the TRRReader on its own seems to take the IOError as the sign to just end iteration. This is bad behavior because one might never learn that one's trajectory is actually damaged and incomplete.

@orbeckst

This comment has been minimized.

Copy link
Member

commented May 24, 2019

It seems that detecting the end of a TRR is pretty haphazard:

# trr are a bit weird. Reading after the last frame always always
# results in an integer error while reading. I tried it also with trr
# produced by different codes (Gromacs, ...).
if return_code != EOK and return_code != EENDOFFILE \
and return_code != EINTEGER:
raise IOError('TRR read error = {}'.format(
error_message[return_code]))
# In a trr the integer error seems to indicate that the file is ending.
# There might be corrupted files where this is a legitimate error. But
# then we just can't read it and stop there which is not too bad.
if return_code == EENDOFFILE or return_code == EINTEGER:
self.reached_eof = True
raise StopIteration
if return_code == EOK:
self.current_frame += 1

However, the reported error here is float and not integer. Not sure why iteration finishes apparently cleanly.

@orbeckst

This comment has been minimized.

Copy link
Member

commented May 24, 2019

I think the reason why "end of file" always shows up as a "EINTEGER" is because the reading of the magic integer fails

if (xdrfile_read_int(&magic, 1, xd) != 1)
return exdrINT;

We could make a special exception that catches that particular case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.