Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extremely slow repr() on small data frame #1746

Closed
njsmith opened this issue Aug 8, 2012 · 3 comments
Closed

Extremely slow repr() on small data frame #1746

njsmith opened this issue Aug 8, 2012 · 3 comments
Labels
Performance Memory or execution speed performance
Milestone

Comments

@njsmith
Copy link

njsmith commented Aug 8, 2012

I have a small data frame here (obtained by slicing a much larger data frame with a MultiIndex):

In [6]: x.shape
Out[6]: (4, 32)

Its repr() looks like this:

In [7]: print repr(x)
<class 'pandas.core.frame.DataFrame'>
MultiIndex: 4 entries, ('arquant9.crw', 10, 336640) to ('arquant9.crw', 10, 336652)
Data columns:
lle     4  non-null values
lhz     4  non-null values
MiPf    4  non-null values
LLPf    4  non-null values
RLPf    4  non-null values
LMPf    4  non-null values
RMPf    4  non-null values
LDFr    4  non-null values
RDFr    4  non-null values
LLFr    4  non-null values
RLFr    4  non-null values
LMFr    4  non-null values
RMFr    4  non-null values
LMCe    4  non-null values
RMCe    4  non-null values
MiCe    4  non-null values
MiPa    4  non-null values
LDCe    4  non-null values
RDCe    4  non-null values
LDPa    4  non-null values
RDPa    4  non-null values
LMOc    4  non-null values
RMOc    4  non-null values
LLTe    4  non-null values
RLTe    4  non-null values
LLOc    4  non-null values
RLOc    4  non-null values
MiOc    4  non-null values
A2      4  non-null values
HEOG    4  non-null values
rle     4  non-null values
rhz     4  non-null values
dtypes: float64(32)

Producing this repr takes a bizarrely long time, it's very noticeable in interactive use:

In [8]: time repr(x)
CPU times: user 1.39 s, sys: 0.06 s, total: 1.45 s
Wall time: 1.36 s

The object is available here: http://vorpus.org/~njs/tmp/slow-repr-x.pickle

@lodagro
Copy link
Contributor

lodagro commented Aug 8, 2012

Seems related to the index

In [68]: time repr(x)
CPU times: user 3.91 s, sys: 0.10 s, total: 4.01 s
Wall time: 3.99 s
...

In [69]: time repr(x.reset_index())
CPU times: user 0.02 s, sys: 0.01 s, total: 0.03 s
Wall time: 0.03 s

@lodagro
Copy link
Contributor

lodagro commented Aug 9, 2012

Some profiling and a little code change ...

In [11]: time repr(x)
CPU times: user 0.03 s, sys: 0.00 s, total: 0.03 s
Wall time: 0.04 s

@lodagro lodagro closed this as completed in b956be0 Aug 9, 2012
@njsmith
Copy link
Author

njsmith commented Aug 9, 2012

Fabulous, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Performance Memory or execution speed performance
Projects
None yet
Development

No branches or pull requests

2 participants