univariate_selection.py:92: RuntimeWarning: invalid value encountered in divide f = msb / msw

[arnevd]

I always seem to get the same error when wanting to use univariate_selection.

univariate_selection.py:92: RuntimeWarning: invalid value encountered in divide f = msb / msw

I am using python 2.7.3 64 bit version and numpy 1.6.2.

The train array I use has the shape (957L, 3317L). Which means there are currently 3317 features and I would like to select the 10% most useful ones using 'SelectPercentile'.

The arrays msb and msw are looking as follows:

msb : ndarray: [-0.01144492 -0.00071531 -0.00643777 ..., -0.00643777 -0.00643777
-0.00286123] with dtype: float64
msw: ndarray: [ 0.00574713 0.00143678 0.00431034 ..., 0.00431034 0.00431034\n 0.00287356] with dtype: float64

I was first running python in 32 bit-modus so I figured that would've been the problem, but after removing it and installing the 64 bit version and all 64 bit extention packages I am still experiencing the same problem.

[amueller]

Thanks for the report @arnevd.
What version of scikit-learn are you using?
It looks to me as if there is some devision by zero because one of the features is completely zero (or constant?). The estimator is not robust against that.
As a quick fix you could try to remove constant features (np.std(X, axis=0) == 0) before calling the function.

[arnevd]

Thanks for the quick response!

I did have some features that occured nowhere. I removed all those and the f array looks just fine. The next problem I have though is that the calculation of prob (see below) results in an array of NaN's.

prob = stats.fprob(dfbn, dfwn, f)

The three variables in the calculation look like this:

f = ndarray: [-1.99373041 -0.4984326 -1.99373041 ..., -1.49529781 -1.49529781
-1.49529781]
dfbn = 2 (because I have three classes; -1, 0 and 1)
dfwn = 954 (becasue I have 957 samples)

Anything I did wrong?

[amueller]

Err... I'm not so familiar with this part of the code.
I won't have time to have a look today, sorry. Not sure who is the expert on univariate feature selection here.
@larsmans probably knows something ;)
We should definitely rename the variables there, that is pretty hard to read...

[arnevd]

Thanks for the quick answer, again.

If I use chi2 instead of f_classif everything works. So I'll just continue using that one. Still don't know what the issue was though

[amueller]

Just remember that chi2 only works for positive data.

[arnevd]

All my features are positive, though one of my classes isn't. Should that result into problems?

[amueller]

No, shouldn't.

[arnevd]

Thanks a lot man!

[amueller]

np. you could help us by providing a minimalistic dataset and script to produce the second problem so we can fix it.

arnevd notifications@github.com schrieb:

Thanks a lot man!

---

Reply to this email directly or view it on GitHub:
#1469 (comment)

[shubhankrr]

I was facing a similar issue and when I changed to chi2 It seems to work. To cross-validate, I took multiple samples and ran chi2 and got an almost similar result among the groups. I also checked for constant variables or variables where all my rows are either zero or NaN and I couldn't find any of it. Not sure exactly what's the problem when we use f_classif.

[DongningS]

I encounter the same problem when building a classification model.

My training dataset originally include both numerical and categorical features. Thus, after numerical transformation of certain categorical features I had discrete numerical values.

Considering the statistic definition relationship between chi2 and f score, chi2is the limiting distribution of the fscore as the denominator degrees of freedom goes to infinity.

chi2: I think choosing chi2 is both a technological feasible and theoretically viable approach for univariate feature selection when we have mixed variables for classification.