Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validation errors caused by incomplete tar file #57

Closed
dmkent opened this issue Aug 7, 2017 · 0 comments

Comments

@dmkent
Copy link
Contributor

@dmkent dmkent commented Aug 7, 2017

I recently hit an issue where an exception was unhandled when validating a file that
was, for unknown reasons, not a valid tar file. Traceback was:

    Traceback (most recent call last):
      File "/miniconda/envs/conda-mirror/lib/python3.5/site-packages/conda_mirror/conda_mirror.py", line 303, in _validate
        t.extractfile('info/index.json').read().decode('utf-8')
      File "/miniconda/envs/conda-mirror/lib/python3.5/tarfile.py", line 2066, in extractfile
        tarinfo = self.getmember(member)
      File "/miniconda/envs/conda-mirror/lib/python3.5/tarfile.py", line 1741, in getmember
        tarinfo = self._getmember(name)
      File "/miniconda/envs/conda-mirror/lib/python3.5/tarfile.py", line 2321, in _getmember
        members = self.getmembers()
      File "/miniconda/envs/conda-mirror/lib/python3.5/tarfile.py", line 1752, in getmembers
        self._load()        # all members, we first have to
      File "/miniconda/envs/conda-mirror/lib/python3.5/tarfile.py", line 2344, in _load
        tarinfo = self.next()
      File "/miniconda/envs/conda-mirror/lib/python3.5/tarfile.py", line 2275, in next
        self.fileobj.seek(self.offset - 1)
      File "/miniconda/envs/conda-mirror/lib/python3.5/bz2.py", line 277, in seek
        return self._buffer.seek(offset, whence)
      File "/miniconda/envs/conda-mirror/lib/python3.5/_compression.py", line 143, in seek
        data = self.read(min(io.DEFAULT_BUFFER_SIZE, offset))
      File "/miniconda/envs/conda-mirror/lib/python3.5/_compression.py", line 99, in read
        raise EOFError("Compressed file ended before the "
    EOFError: Compressed file ended before the end-of-stream marker was reached

There seems to have been something wrong with the file downloaded – have yet to reproduce what. Possibly our caching proxy helped make things worse…

However, the file should have failed validation, been removed and conda-mirror continued rather than giving up completely.

Will submit a small patch that resolves this by handling EOFError.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.