Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: savez raises exception for masked arrays #22728

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

jpbianchi
Copy link

Resolves #18134
savez() now raises an exception for a masked array, like save() already did.
Tests were added for both functions to test the exception being raised in case a masked array is passed as argument.

numpy/ma/tests/test_core.py Outdated Show resolved Hide resolved
numpy/ma/tests/test_core.py Outdated Show resolved Hide resolved
jpbianchi and others added 2 commits December 3, 2022 20:32
Co-authored-by: Rohit Goswami <r95g10@gmail.com>
Co-authored-by: Rohit Goswami <r95g10@gmail.com>
Copy link
Member

@HaoZeke HaoZeke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there :)

numpy/ma/tests/test_core.py Outdated Show resolved Hide resolved
Co-authored-by: Rohit Goswami <r95g10@gmail.com>
Copy link
Member

@HaoZeke HaoZeke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passes the new test.

Comment on lines +51 to +52
return isinstance(f, (zipfile._ZipWriteFile, io.FileIO,
io.BufferedReader, io.BufferedWriter))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will now break the tests depending on this tuple. e.g.:

__________________________ test_compressed_roundtrip ___________________________

tmpdir = local('/tmp/pytest-of-runner/pytest-0/test_compressed_roundtrip0')

    def test_compressed_roundtrip(tmpdir):
        arr = np.random.rand(200, 200)
        npz_file = os.path.join(tmpdir, 'compressed.npz')
>       np.savez_compressed(npz_file, arr=arr)

arr        = array([[0.75950489, 0.26798953, 0.83152339, ..., 0.907[3414](https://github.com/numpy/numpy/actions/runs/3609969600/jobs/6083480553#step:4:3418)2, 0.76341456,
        0.41962783],
       [0.71932335, 0.65...59,
        0.22654725],
       [0.17476853, 0.4739831 , 0.52077257, ..., 0.97867451, 0.50465821,
        0.2018301 ]])
npz_file   = '/tmp/pytest-of-runner/pytest-0/test_compressed_roundtrip0/compressed.npz'
tmpdir     = local('/tmp/pytest-of-runner/pytest-0/test_compressed_roundtrip0')

../../builds/venv/lib/python3.8/site-packages/numpy-1.25.0.dev0+109.g713be80dd-py3.8-linux-x86_64.egg/numpy/lib/tests/test_format.py:497:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other examples:

Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
    self.close()
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
    self.fp.seek(self.start_dir)
ValueError: I/O operation on closed file.
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
    self.close()
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
    self.fp.seek(self.start_dir)
ValueError: I/O operation on closed file.
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
    self.close()
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
    self.fp.seek(self.start_dir)
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/tempfile.py", line 473, in func_wrapper
    return func(*args, **kwargs)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
    self.close()
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
    self.fp.seek(self.start_dir)
ValueError: I/O operation on closed file.
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
    self.close()
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
    self.fp.seek(self.start_dir)
ValueError: I/O operation on closed file.
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
    self.close()
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
    self.fp.seek(self.start_dir)
ValueError: I/O operation on closed file.
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first error comes from the fact that, after my change, the zipfile type is checked by "if isfileobj(fp)", and then the command 'array.tofile()' is executed, which was not the case before.
I found out that that instruction works for maskedarrays, but NOT for arrays (which is what the test 'test_compressed_roundtrip' uses).
I've reproduced the issue by stopping the debugger at that instruction, and tested both arrays and masked_arrays, and you can see that arrays give an error about using fileno() while masked_arrays give the exception we want.
debugger screenshot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Awaiting a code review
Development

Successfully merging this pull request may close these issues.

BUG: Masked array can be saved to file with numpy.savez(), but after loading it, the mask is gone.
3 participants