Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PERF/API/BUG: index memory not releasing when manipulated by numpy functions #8880

Closed
jreback opened this issue Nov 22, 2014 · 1 comment
Closed
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Performance Memory or execution speed performance

Comments

@jreback
Copy link
Contributor

jreback commented Nov 22, 2014

http://stackoverflow.com/questions/27074469/memory-leak-in-python-pandas-reshuffling-index

In [1]: df = DataFrame(np.random.randn(10000))

In [2]: def f(df):
   ...:     for dfi in np.array_split(df,100):
   ...:         shuffle = dfi.reindex(np.random.permutation(dfi.index))
   ...:         one = shuffle.iloc[:50]
   ...:         two = shuffle.iloc[50:]

so the np.random.permutation seems to be shuffling the index in a weird way
this doesn't leak with dfi.index.values

@jreback jreback added Bug Indexing Related to indexing on series/frames, not to indexes themselves Performance Memory or execution speed performance API Design labels Nov 22, 2014
@jreback jreback added this to the 0.16.0 milestone Nov 22, 2014
@jreback jreback modified the milestones: 0.16.0, Next Major Release Mar 6, 2015
@mroeschke mroeschke removed this from the Contributions Welcome milestone Oct 13, 2022
@mroeschke
Copy link
Member

Don't see this on main anymore so it must have been fixed. Closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Performance Memory or execution speed performance
Projects
None yet
Development

No branches or pull requests

2 participants