Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
ENH Enable bzip2 streaming for Python 3 #11072
Conversation
|
tests! |
stephen-hoover
commented
Sep 12, 2015
|
I added a test for reading from an open file with the C parser. It fails on the master branch and passes here. How's that? |
|
do you have exactly the same deps |
stephen-hoover
commented
Sep 12, 2015
|
Yes, exactly the same dependencies. This PR works because the standard library |
jreback
added Data IO CSV
labels
Sep 12, 2015
|
ok, this looks good. pls add a note in whatsnew for 0.17.0 (just released the rc1 yesterday, but this is ok). reference both the original issue and this PR number I think. squash & ping when green. |
jreback
added this to the
0.17.0
milestone
Sep 12, 2015
stephen-hoover
commented
Sep 12, 2015
|
Note added. It doesn't look like anything else references a PR; should I leave that reference in? |
jreback
commented on an outdated diff
Sep 12, 2015
| @@ -465,6 +465,8 @@ Other enhancements | ||
| - Improved error message when concatenating an empty iterable of dataframes (:issue:`9157`) | ||
| +- ``pd.read_csv`` can now read bz2-compressed files incrementally, and the C parser can read bz2-compressed files from AWS S3 (:issue:`110701`, :pr:`11072`). |
jreback
Contributor
|
stephen-hoover
referenced
this pull request
Sep 12, 2015
Closed
Improvements for read_csv from AWS S3 #11070
stephen-hoover
commented
Sep 13, 2015
|
@jreback , tests are green! |
jreback
added a commit
that referenced
this pull request
Sep 13, 2015
|
|
jreback |
e8d4243
|
jreback
merged commit e8d4243
into pandas-dev:master
Sep 13, 2015
1 check passed
|
thanks! |
stephen-hoover commentedSep 12, 2015
This is the one modification related to issue #11070 which affects non-S3 interactions with
read_csv. The Python 3 standard library has an improved capability for handling bz2 compression, so a simple change will letread_csvstream bz2-compressed files.