Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Pandas .fillna() should handle "inf" #2858

Closed
darindillon opened this Issue Feb 12, 2013 · 3 comments

Comments

Projects
None yet
4 participants

Division by 0 in pandas will give the value "inf". But the .fillna() method doesn't recognize that. We should make .fillna() handle "inf" the same way it handles "NaN'. (for reference, the numpy.isfinite() method treats NaN and Inf interchangably -- pandas should do the same).

import pandas
vals = [ 1.1, 1.1, 1.1]
p = pandas.DataFrame( { 'first' : vals }, columns=['first'])
p['first'] = p['first'] / 0 #Creates a bunch of "inf" values
p['first'].fillna(0, inplace=True) #Has no effect

Contributor

y-p commented Feb 12, 2013

Possibly you're looking for

In [37]: pd.describe_option("use_inf_as_null")
mode.use_inf_as_null: 
: boolean
        True means treat None, NaN, INF, -INF as null (old way),
        False means None and NaN are null, but INF, -INF are not null
        (new way).

GH tip: you can quote your snippets to get syntax highlighting. pretty colors.

Owner

wesm commented Feb 12, 2013

In [2]: p
Out[2]: 
   first
0    inf
1    inf
2    inf

In [3]: p.replace(np.inf, 0)
Out[3]: 
   first
0      0
1      0
2      0

@wesm and then you have to remember 'method' otherwise it silently fails... nvm, long story
@tavistmorph this whole issue highlights how unpythonic the current implementation is. Adding a kwarg to include np.inf and -np.inf would've been much easier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment