BUG: fix scipy.io.mmread() of gzipped files under Python3 #3314
Conversation
|
Changes Unknown when pulling 8daf621 on andreas-h:gh2152 into * on scipy:master*. |
|
Changes Unknown when pulling 8daf621 on andreas-h:gh2152 into * on scipy:master*. |
|
Changes Unknown when pulling 8daf621 on andreas-h:gh2152 into * on scipy:master*. |
|
Changes Unknown when pulling 8daf621 on andreas-h:gh2152 into * on scipy:master*. |
| @@ -451,9 +452,14 @@ def _parse_body(self, stream): | |||
| return coo_matrix((rows, cols), dtype=dtype) | |||
|
|
|||
| try: | |||
| # passing a gzipped file to fromfile/fromstring doesn't work | |||
| # with Python3 | |||
| if sys.version_info >= (3, 0) and isinstance(stream, gzip.GzipFile): | |||
There was a problem hiding this comment.
probably the same occurs with bz2, zip and xz files
There was a problem hiding this comment.
numpy uses np.compat.isfileobj to determine if fromfile can be used
There was a problem hiding this comment.
on second thought that function is too restrictive it rejects io based file objects which are default in python3
There was a problem hiding this comment.
ignore the last comment, I opened files in write modes, it seems to work if they are binary read mode.
|
added checking for bzip2 files. couldn't find xz support in mmread at all. |
|
Test? |
|
can we assume that gzip and bz2 modules are available on all platforms? if not, how can I mark the tests accordingly? |
|
I just saw that Py3.3 has a lzma module. Should we add support for |
|
AFAIK at least the bz2 module is not necessarily present in all Python builds. |
|
The |
| def test_bzip2_py3(self): | ||
| # test if fix for #2152 works | ||
| try: | ||
| import bz2 |
There was a problem hiding this comment.
this needs a comment on when the module is/isn't available
There was a problem hiding this comment.
Can you be more specific? I'm not sure how such things are handled in
the tests.
There was a problem hiding this comment.
Just above the import I would just put # bz2 module isn't always built when building Python. Otherwise one always wonders why the import is guarded. The stdlib docs at http://docs.python.org/2/library/bz2.html don't even explain this:(
There was a problem hiding this comment.
Shall I add the same note to the gzip import? Or is gzip always present
and I can remove the try/except?
There was a problem hiding this comment.
No, gzip can also be missing. Adding same note makes sense.
|
removing needs-work label, looks like previous comments were addressed. |
BUG: fix scipy.io.mmread() of gzipped files under Python3
|
Merging, looks fine now. Thanks @andreas-h |
|
One remaining buglet fixed in 86d3722 (pushed to master). I don't really like checking for "bad classes" manually, but on the other hand, I'm not sure if it's even possible to do this checking in a generic way. |
|
Thanks |
fixes #2152