Infer compression if file extension is uppercase #35164
Conversation
|
Do we do this elsewhere? Outside of Windows most file systems I think are case sensitive so while relatively harmless I also am not sure this is worth doing |
|
I think the path is retained with the original case and couldn't see any uses beyond identifying the compression method. Ran into this issue on OS X, tested locally with |
gfyoung
added
IO Data
Not that I'm aware of.
I think it would be okay to add this, but only after checking if we are on Windows. cc @jreback |
doc/source/whatsnew/v1.1.0.rst
Outdated
| @@ -1050,6 +1050,7 @@ I/O | |||
| - Bug in :meth:`~HDFStore.create_table` now raises an error when `column` argument was not specified in `data_columns` on input (:issue:`28156`) | |||
| - :meth:`read_json` now could read line-delimited json file from a file url while `lines` and `chunksize` are set. | |||
| - Bug in :meth:`DataFrame.to_sql` when reading DataFrames with ``-np.inf`` entries with MySQL now has a more explicit ``ValueError`` (:issue:`34431`) | |||
| - Bug in :meth:`io.common.infer_compression` where capitalised files extensions were not decompressed by read_* functions. | |||
There was a problem hiding this comment.
can you add this PR number here (use the issue format). Also no need to reference this internal function at all.
|
Does pathlib offer any abstractions here? I don’t think it’s worthwhile for pandas to implement its own rules for file name equivalency.
…Sent from my iPhone
On Jul 9, 2020, at 3:02 PM, Jeff Reback ***@***.***> wrote:
@jreback requested changes on this pull request.
i think this is ok to do everywhere.
In doc/source/whatsnew/v1.1.0.rst:
> @@ -1050,6 +1050,7 @@ I/O
- Bug in :meth:`~HDFStore.create_table` now raises an error when `column` argument was not specified in `data_columns` on input (:issue:`28156`)
- :meth:`read_json` now could read line-delimited json file from a file url while `lines` and `chunksize` are set.
- Bug in :meth:`DataFrame.to_sql` when reading DataFrames with ``-np.inf`` entries with MySQL now has a more explicit ``ValueError`` (:issue:`34431`)
+- Bug in :meth:`io.common.infer_compression` where capitalised files extensions were not decompressed by read_* functions.
can you add this PR number here (use the issue format). also add 'on windows platforms'
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
this is fine; we already have our own rules for when to decompress - this is just expanding to capitalized which is not a big deal |
|
thanks @willbowditch |
Inferring compression fails for files with uppercase extensions (e.g.
x.zipworks buty.ZIPdoes not)black pandasgit diff upstream/master -u -- "*.py" | flake8 --diff