Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strange behaviour of pd.DataFrame.drop() with inplace argument #30484

Closed
DenisVorotyntsev opened this issue Dec 26, 2019 · 5 comments · Fixed by #30501
Closed

Strange behaviour of pd.DataFrame.drop() with inplace argument #30484

DenisVorotyntsev opened this issue Dec 26, 2019 · 5 comments · Fixed by #30501
Labels
Bug Copy / view semantics inplace Relating to inplace parameter or equivalent
Milestone

Comments

@DenisVorotyntsev
Copy link

DenisVorotyntsev commented Dec 26, 2019

Code Sample, a copy-pastable example if possible

# input - 1 
df = pd.DataFrame({})
df["x1"] = [1, 2, 3, 4, 5]
df["x2"] = [0, 0, 0, 1, 1]
df["target"] = [10, 20, 30, 40, 50]
y = df["target"]
df.drop("target", axis=1, inplace=True)
y = y + np.min(y)

# output df (without removed target column)
	x1	x2
0	1	0
1	2	0
2	3	0
3	4	1
4	5	1

# input - 2 
df = pd.DataFrame({})
df["x1"] = [1, 2, 3, 4, 5]
df["x2"] = [0, 0, 0, 1, 1]
df["target"] = [10, 20, 30, 40, 50]
y = df["target"]
df.drop("target", axis=1, inplace=True)
y += np.min(y)

# output df (with the removed target column)

  | x1 | x2 | target
-- | -- | -- | --
0 | 1 | 0 | 20
1 | 2 | 0 | 30
2 | 3 | 0 | 40
3 | 4 | 1 | 50
4 | 5 | 1 | 60

Problem description

Pandas return DataFrame with removed column after doing in-place operation with a column. Is it a bug or intentional behavior?

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None

pandas : 0.25.1
numpy : 1.16.2
pytz : 2018.9
dateutil : 2.7.5
pip : 18.1
setuptools : 42.0.2
Cython : 0.29.2
pytest : 4.1.0
hypothesis : None
sphinx : 1.8.3
blosc : None
feather : None
xlsxwriter : 1.1.2
lxml.etree : 4.3.0
html5lib : 1.0.1
pymysql : None
psycopg2 : None
jinja2 : 2.10
IPython : 7.2.0
pandas_datareader: None
bs4 : 4.7.1
bottleneck : 1.2.1
fastparquet : None
gcsfs : None
lxml.etree : 4.3.0
matplotlib : 3.0.2
numexpr : 2.6.9
odfpy : None
openpyxl : 2.5.12
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : 1.2.1
sqlalchemy : 1.2.15
tables : 3.4.4
xarray : 0.14.0
xlrd : 1.2.0
xlwt : 1.3.0
xlsxwriter : 1.1.2

@WillAyd
Copy link
Member

WillAyd commented Dec 26, 2019

The use of the inplace keyword is strongly discouraged. If you replace that with the more idiomatic df = df.drop("target", axis=1) then things look OK

If you'd like to take a look internally would welcome a patch

@WillAyd WillAyd added the Bug label Dec 26, 2019
@WillAyd WillAyd changed the title Strange behaviour of pd.DataFrame.drop() Strange behaviour of pd.DataFrame.drop() with inplace argument Dec 26, 2019
@DenisVorotyntsev
Copy link
Author

df = df.drop("target", axis=1)
Does it require more memory than inplace?

@FSpanhel
Copy link

Regarding #30484 (comment) and #30484 (comment).

I would also not recommend the use of the inplace keyword. Note however, that you may have to adjust your code because df = df.drop("target", axis=1) returns a copy and not a view. See #33438.

@jorisvandenbossche
Copy link
Member

Reopening since we are reverting the original fix because it caused another regression (see #36373)

@jorisvandenbossche jorisvandenbossche modified the milestones: 1.1, Contributions Welcome Oct 29, 2020
@mroeschke mroeschke removed the Reshaping Concat, Merge/Join, Stack/Unstack, Explode label Jul 25, 2021
@jbrockmendel jbrockmendel added the inplace Relating to inplace parameter or equivalent label Oct 29, 2021
@mroeschke mroeschke removed this from the Contributions Welcome milestone Oct 13, 2022
@jorisvandenbossche
Copy link
Member

In the end we actually didn't revert the fix, but merged an alternative PR for the regression that preserved the fix for this issue: #37508. So closing again.

@jorisvandenbossche jorisvandenbossche added this to the 1.1 milestone Oct 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Copy / view semantics inplace Relating to inplace parameter or equivalent
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants