New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLN: consolidate compression inference as much as possible #12688

Closed
jreback opened this Issue Mar 22, 2016 · 5 comments

Comments

Projects
None yet
4 participants
@jreback
Contributor

jreback commented Mar 22, 2016

we have 3 copies of the compression inference routines in parser.py, parser.py, and io/common.py. See how much we can reduce this (some is necessary as sometimes we need to actually return a file handle or need to read in and re-wrap).

xref #12175

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Mar 22, 2016

Contributor
Contributor

jreback commented Mar 22, 2016

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback May 31, 2016

Contributor

@gfyoung if you are looking for something to do :>

I think we can blow away _wrap_compresed in parsers/io/parsers.py and incorporate any such functionaility in io/common.py/_get_handle

there is also compression handling in parser.pyx but I think this might be harder to remove (in favor of _get_handle, though maybe if we pass an argument might work).

xref #13317

Contributor

jreback commented May 31, 2016

@gfyoung if you are looking for something to do :>

I think we can blow away _wrap_compresed in parsers/io/parsers.py and incorporate any such functionaility in io/common.py/_get_handle

there is also compression handling in parser.pyx but I think this might be harder to remove (in favor of _get_handle, though maybe if we pass an argument might work).

xref #13317

@gfyoung

This comment has been minimized.

Show comment
Hide comment
@gfyoung

gfyoung May 31, 2016

Member

Haha. Thanks @jreback for the mention. I'll take a closer look when I have the time. 😄

Member

gfyoung commented May 31, 2016

Haha. Thanks @jreback for the mention. I'll take a closer look when I have the time. 😄

@lababidi

This comment has been minimized.

Show comment
Hide comment
@lababidi

lababidi May 31, 2016

@jreback I got the following error when attempting to test:

TypeError: assert_almost_equal() got an unexpected keyword argument 'check_dtype'```

lababidi commented May 31, 2016

@jreback I got the following error when attempting to test:

TypeError: assert_almost_equal() got an unexpected keyword argument 'check_dtype'```
@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback May 31, 2016

Contributor

you need to rebuild extensions

python setup.py build_ext --inplace

Contributor

jreback commented May 31, 2016

you need to rebuild extensions

python setup.py build_ext --inplace

@lababidi lababidi referenced this issue Jun 1, 2016

Closed

Cleanup compression #13340

4 of 4 tasks complete

@jorisvandenbossche jorisvandenbossche modified the milestones: Next Major Release, 0.19.0 Sep 1, 2016

dhimmel added a commit to dhimmel/pandas that referenced this issue Dec 13, 2016

dhimmel added a commit to dhimmel/pandas that referenced this issue Dec 13, 2016

@jreback jreback closed this in 3761448 Dec 13, 2016

@jreback jreback modified the milestones: 0.20.0, Next Major Release Dec 13, 2016

ischurov added a commit to ischurov/pandas that referenced this issue Dec 19, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment