Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extract_archive: support deflate64-compressed zip files #282

Merged
merged 8 commits into from
Jan 15, 2022

Conversation

adamjstewart
Copy link
Collaborator

@adamjstewart adamjstewart commented Dec 14, 2021

Fixes #59

@adamjstewart adamjstewart added the datasets Geospatial or benchmark datasets label Dec 14, 2021
@@ -81,6 +80,26 @@ def __exit__(self, exc_type: None, exc_value: None, traceback: None) -> None:
pass


class _zipfile:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we didn't need test coverage, this entire class could be replaced by:

try:
    import zipfile_deflate64 as zipfile
except ImportError:
    import zipfile

If zipfile_deflate64 is not installed and a user opens a deflate64-compressed zip file, they'll see the error message:

Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "/Users/Adam/.spack/.spack-env/._view/tmizjnr2yebouizawyxzmo3gwignw5lz/lib/python3.8/zipfile.py", line 1647, in extractall
    self._extract_member(zipinfo, path, pwd)
  File "/Users/Adam/.spack/.spack-env/._view/tmizjnr2yebouizawyxzmo3gwignw5lz/lib/python3.8/zipfile.py", line 1700, in _extract_member
    with self.open(member, pwd=pwd) as source, \
  File "/Users/Adam/.spack/.spack-env/._view/tmizjnr2yebouizawyxzmo3gwignw5lz/lib/python3.8/zipfile.py", line 1571, in open
    return ZipExtFile(zef_file, mode, zinfo, pwd, True)
  File "/Users/Adam/.spack/.spack-env/._view/tmizjnr2yebouizawyxzmo3gwignw5lz/lib/python3.8/zipfile.py", line 819, in __init__
    self._decompressor = _get_decompressor(self._compress_type)
  File "/Users/Adam/.spack/.spack-env/._view/tmizjnr2yebouizawyxzmo3gwignw5lz/lib/python3.8/zipfile.py", line 720, in _get_decompressor
    _check_compression(compress_type)
  File "/Users/Adam/.spack/.spack-env/._view/tmizjnr2yebouizawyxzmo3gwignw5lz/lib/python3.8/zipfile.py", line 700, in _check_compression
    raise NotImplementedError("That compression method is not supported")
NotImplementedError: That compression method is not supported

We could potentially catch this error and add a more helpful error message like we do with rarfile, but it isn't clear how to test this (see PR description).

@adamjstewart adamjstewart added this to the 0.1.1 milestone Dec 14, 2021
@adamjstewart adamjstewart marked this pull request as draft December 14, 2021 05:38
@adamjstewart
Copy link
Collaborator Author

This library might actually be broken: brianhelba/zipfile-deflate64#19

@adamjstewart adamjstewart modified the milestones: 0.1.1, 0.1.2 Dec 19, 2021
@adamjstewart adamjstewart modified the milestones: 0.1.2, 0.2.1 Jan 1, 2022
@github-actions github-actions bot added the testing Continuous integration testing label Jan 5, 2022
@adamjstewart adamjstewart marked this pull request as ready for review January 5, 2022 16:12
@adamjstewart adamjstewart reopened this Jan 6, 2022
@adamjstewart
Copy link
Collaborator Author

Can't tell if RtD is broken or this PR is broken, but I'm going to guess the former.

@adamjstewart adamjstewart merged commit f32bc0c into main Jan 15, 2022
@adamjstewart adamjstewart deleted the fixes/zipfile-deflate64 branch January 15, 2022 05:14
yichiac pushed a commit to yichiac/torchgeo that referenced this pull request Apr 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datasets Geospatial or benchmark datasets testing Continuous integration testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Chesapeake datasets don't extract properly
2 participants