New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG: savez raises exception for masked arrays #22728
base: main
Are you sure you want to change the base?
Conversation
6392364
to
b9b32b0
Compare
Co-authored-by: Rohit Goswami <r95g10@gmail.com>
Co-authored-by: Rohit Goswami <r95g10@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Almost there :)
Co-authored-by: Rohit Goswami <r95g10@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Passes the new test.
return isinstance(f, (zipfile._ZipWriteFile, io.FileIO, | ||
io.BufferedReader, io.BufferedWriter)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will now break the tests depending on this tuple. e.g.:
__________________________ test_compressed_roundtrip ___________________________
tmpdir = local('/tmp/pytest-of-runner/pytest-0/test_compressed_roundtrip0')
def test_compressed_roundtrip(tmpdir):
arr = np.random.rand(200, 200)
npz_file = os.path.join(tmpdir, 'compressed.npz')
> np.savez_compressed(npz_file, arr=arr)
arr = array([[0.75950489, 0.26798953, 0.83152339, ..., 0.907[3414](https://github.com/numpy/numpy/actions/runs/3609969600/jobs/6083480553#step:4:3418)2, 0.76341456,
0.41962783],
[0.71932335, 0.65...59,
0.22654725],
[0.17476853, 0.4739831 , 0.52077257, ..., 0.97867451, 0.50465821,
0.2018301 ]])
npz_file = '/tmp/pytest-of-runner/pytest-0/test_compressed_roundtrip0/compressed.npz'
tmpdir = local('/tmp/pytest-of-runner/pytest-0/test_compressed_roundtrip0')
../../builds/venv/lib/python3.8/site-packages/numpy-1.25.0.dev0+109.g713be80dd-py3.8-linux-x86_64.egg/numpy/lib/tests/test_format.py:497:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other examples:
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
self.close()
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
self.fp.seek(self.start_dir)
ValueError: I/O operation on closed file.
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
self.close()
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
self.fp.seek(self.start_dir)
ValueError: I/O operation on closed file.
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
self.close()
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
self.fp.seek(self.start_dir)
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/tempfile.py", line 473, in func_wrapper
return func(*args, **kwargs)
ValueError: seek of closed file
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
self.close()
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
self.fp.seek(self.start_dir)
ValueError: I/O operation on closed file.
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
self.close()
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
self.fp.seek(self.start_dir)
ValueError: I/O operation on closed file.
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1821, in __del__
self.close()
File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/zipfile.py", line 1838, in close
self.fp.seek(self.start_dir)
ValueError: I/O operation on closed file.
Exception ignored in: <function ZipFile.__del__ at 0x7fc089e4d1f0>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The first error comes from the fact that, after my change, the zipfile type is checked by "if isfileobj(fp)", and then the command 'array.tofile()' is executed, which was not the case before.
I found out that that instruction works for maskedarrays, but NOT for arrays (which is what the test 'test_compressed_roundtrip' uses).
I've reproduced the issue by stopping the debugger at that instruction, and tested both arrays and masked_arrays, and you can see that arrays give an error about using fileno() while masked_arrays give the exception we want.
debugger screenshot
Resolves #18134
savez() now raises an exception for a masked array, like save() already did.
Tests were added for both functions to test the exception being raised in case a masked array is passed as argument.