Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

diff() behavior causes memory problems #2266

Closed
etyurin opened this issue Nov 15, 2012 · 3 comments

Comments

@etyurin
Copy link

commented Nov 15, 2012

diff(n) with a negative n used to be a recipe for obtaining a forward-looking diff. Now it causes memory problems.

The change was introduced in commit 06f74e5 and "improved" in commit 0d6fb1b381ef3882ceb766fecb0e28a5d5c2bd2bw . The first commit produces unexpected results silently. The second commit causes an outright coredump with

*** glibc detected *** /usr/bin/python: double free or corruption (out): 0x000000000591db00 ***

This is a simple script to trigger this bug:

import pandas
import numpy as np
x = np.sin(np.arange(24))
x = x.reshape((4,6))
df = pandas.DataFrame(x,columns=['a','b','c','d','e','f'])
print df
df2 = -df.diff(-1)
print df2

The results used to be mathematically correct:

      a         b         c         d         e         f

0 0.000000 0.841471 0.909297 0.141120 -0.756802 -0.958924
1 -0.279415 0.656987 0.989358 0.412118 -0.544021 -0.999990
2 -0.536573 0.420167 0.990607 0.650288 -0.287903 -0.961397
3 -0.750987 0.149877 0.912945 0.836656 -0.008851 -0.846220
a b c d e f
0 -0.279415 -0.184484 0.080061 0.270998 0.212781 -0.041066
1 -0.257157 -0.236820 0.001249 0.238169 0.256118 0.038593
2 -0.214414 -0.270290 -0.077662 0.186368 0.279052 0.115177
3 NaN NaN NaN NaN NaN NaN

Now they are not even consistent column-by-column:

      a         b         c         d         e         f

0 0.000000 0.841471 0.909297 0.141120 -0.756802 -0.958924
1 -0.279415 0.656987 0.989358 0.412118 -0.544021 -0.999990
2 -0.536573 0.420167 0.990607 0.650288 -0.287903 -0.961397
3 -0.750987 0.149877 0.912945 0.836656 -0.008851 -0.846220
a b c d e f
0 NaN NaN NaN NaN NaN NaN
1 NaN NaN NaN NaN NaN NaN
2 NaN NaN NaN NaN NaN NaN
3 0.750987 0.691594 -0.003648 -0.695536 -0.747951 -0.112704

@ghost ghost assigned changhiskhan Nov 15, 2012

@changhiskhan

This comment has been minimized.

Copy link
Contributor

commented Nov 15, 2012

we'll put in a fix. thanks for the report

changhiskhan added a commit that referenced this issue Nov 15, 2012
@changhiskhan

This comment has been minimized.

Copy link
Contributor

commented Nov 15, 2012

just put in a fix. Please let us know if you have any additional issues.

@wesm wesm closed this Nov 17, 2012

@wesm

This comment has been minimized.

Copy link
Member

commented Nov 17, 2012

thanks, Chang

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.