SettingWithCopyWarning: possible spurious warning after dropna() call

[jorisvandenbossche]

I get a SettingWithCopyWarning after the use of dropna, which does not really seem logical:

In [102]: df = pd.DataFrame({'a':[1,2,np.nan], 'b':['A', 'B', 'C']})

In [103]: df
Out[103]:
    a  b
0   1  A
1   2  B
2 NaN  C

In [104]: df = df.dropna(subset=['a'])

In [105]: df
Out[105]:
   a  b
0  1  A
1  2  B

In [106]: df['new_col'] = 2
C:\Anaconda\envs\devel\Scripts\ipython-script.py:1: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/s
table/indexing.html#indexing-view-versus-copy
  if __name__ == '__main__':

In [107]: df.loc[:,'new_col'] = 2
c:\users\vdbosscj\scipy\pandas-joris\pandas\core\indexing.py:404: SettingWithCop
yWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/s
table/indexing.html#indexing-view-versus-copy
  self.obj[item] = s

Further, also the indication were the warning occurred is not correct I think.

[jreback]

works on master for me.

In [1]: df = pd.DataFrame({'a':[1,2,np.nan], 'b':['A', 'B', 'C']})

In [2]: df = df.dropna(subset=['a'])

In [3]: df['new_col'] = 2

This would be a setting with copy, except that you are reassigning the reference and it is garbage collected, so its ok.

[jorisvandenbossche]

Whoops, I was accidentally using 0.15.1, didn't check with 0.16.1 (or master). Indeed this does not raise!

[jorisvandenbossche]

For the case you reassign to another name:

df2 = df.dropna(subset=['a'])
df2['new_col'] = 2

it does raise. But personally, I still find this a bit strange. As a user, I wouldn't expect it that my change would be applied on the original df (I expext dropna to take a copy). So in that sense, it is very strange to get this warning. Also, you get it with df2.loc[:, 'new_col'] = 2 as well, where the warnings says to use .loc[...] which I am already doing.

[jreback]

.dropna is just a fancy name for .ix with an inverse indexer

e.g.

In [30]: df.ix[pd.notnull(df.a)]
Out[30]: 
   a  b
0  1  A
1  2  B

In [31]: df.ix[pd.notnull(df.a)].is_copy
Out[31]: <weakref at 0x121aa7f70; to 'DataFrame' at 0x121a75c90>

we could make dropna return a copy, though its really just a slicing operation, hence the warning

[jreback]

.dropna has inplace=False by default, so this might be a bug. why don't you rename, reopen (and change the top example)