GradientBoostingClassifier.fit accepts sparse X, but .predict does not

[aflaxman]

I have a sparse dataset that is too large for main memory if I call X.todense(). If I understand correctly, GradientBoostingClassifier.fit will accept my sparse X, but it is not currently possible to use GradientBoostingClassifier.predict on the results. It would be great if that were not the case.

Here is a minimal example of the issue:

from scipy import sparse
from sklearn.datasets.samples_generator import make_classification
from sklearn.ensemble import GradientBoostingClassifier

X, y = make_classification(n_samples=20, n_features=5, random_state=0)
X_sp = sparse.coo_matrix(X)

clf = GradientBoostingClassifier()
clf.fit(X,y)
clf.predict(X)  # works

clf.fit(X_sp, y)  # works
clf.predict(X_sp)  # fails with TypeError: A sparse matrix was passed, but dense data is required.

[olologin]

Confirmed. I think small rework of https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/ensemble/_gradient_boosting.pyx#L39 is needed.
Looks like function _predict_regression_tree_inplace_fast was written to somehow optimize prediction speed. But i see some problems in it, its main loop https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/ensemble/_gradient_boosting.pyx#L92 differs from the main loop here https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/tree/_tree.pyx#L784, and additionally this function doesn't work with sparse matrices.

Maybe it's better to rely on predict method of Tree class, but some performance tests is needed.

[Sandy4321]

this problem still in
clf.predict(X_sp)
Traceback (most recent call last):
Debug Probe, prompt 11, line 1
File "C:\Anaconda\Lib\site-packages\sklearn\ensemble\gradient_boosting.py", line 1498, in predict
score = self.decision_function(X)
File "C:\Anaconda\Lib\site-packages\sklearn\ensemble\gradient_boosting.py", line 1456, in decision_function
X = check_array(X, dtype=DTYPE, order="C")
File "C:\Anaconda\Lib\site-packages\sklearn\utils\validation.py", line 371, in check_array
force_all_finite)
File "C:\Anaconda\Lib\site-packages\sklearn\utils\validation.py", line 238, in _ensure_sparse_format
raise TypeError('A sparse matrix was passed, but dense '
TypeError: A sparse matrix was passed, but dense data is required. Use X.toarray() to convert to a dense numpy array.

[aflaxman]

@Sandy4321 here is how you can try out the code in PR #6116 :

git clone https://github.com/scikit-learn/scikit-learn.git
cd scikit-learn
git pull https://github.com/olologin/scikit-learn.git GradientBoostingFix
python setup.py install
cd ..

It would be cool if GitHub gave this advice on the PR page... I know it has good advice on how to merge a PR manually if you are a repo admin. But it is the non-admins like me who really need the step-by-step instructions.

BTW, when I do this, my minimal example succeeds! Thanks @olologin !

[Sandy4321]

And I have anaconda python installation, recently I updated scikit by pip
and this destroyed scikit on my computer... It took me many efforts to get
scikit work... Does this have some risk? Maybe better update only one, two
files?
On Jan 14, 2016 4:10 PM, "Abraham Flaxman" notifications@github.com wrote:

@Sandy4321 https://github.com/Sandy4321 here is how you can try out the
code in PR #6116 #6116
:

git clone https://github.com/scikit-learn/scikit-learn.git
cd scikit-learn
git pull https://github.com/olologin/scikit-learn.git GradientBoostingFix
python setup.py install
cd ..

It would be cool if GitHub gave this advice on the PR page... I know it
has good advice on how to merge a PR manually if you are a repo admin. But
it is the non-admins like me who really need the step-by-step instructions.

BTW, when I do this, my minimal example succeeds! Thanks @olologin
https://github.com/olologin !

—
Reply to this email directly or view it on GitHub
#6101 (comment)
.

[Sandy4321]

OK I have windows computer, so all these are commands in dos window?
On Jan 14, 2016 4:10 PM, "Abraham Flaxman" notifications@github.com wrote:

@Sandy4321 https://github.com/Sandy4321 here is how you can try out the
code in PR #6116 #6116
:

git clone https://github.com/scikit-learn/scikit-learn.git
cd scikit-learn
git pull https://github.com/olologin/scikit-learn.git GradientBoostingFix
python setup.py install
cd ..

It would be cool if GitHub gave this advice on the PR page... I know it
has good advice on how to merge a PR manually if you are a repo admin. But
it is the non-admins like me who really need the step-by-step instructions.

BTW, when I do this, my minimal example succeeds! Thanks @olologin
https://github.com/olologin !

—
Reply to this email directly or view it on GitHub
#6101 (comment)
.

[olologin]

@Sandy4321, I'm not sure about windows computer, because you need to install git VCS.

Read here about dev-version building http://scikit-learn.org/stable/developers/advanced_installation.html#install-bleeding-edge

[Sandy4321]

ok,
easy does it, may I just have py file to implement it

On Thu, Jan 14, 2016 at 11:30 PM, Ganiev Ibraim notifications@github.com
wrote:

@Sandy4321 https://github.com/Sandy4321, I'm not sure about windows
computer, because you need to install git VCS.

Read here about dev-version building
http://scikit-learn.org/stable/developers/advanced_installation.html#install-bleeding-edge

—
Reply to this email directly or view it on GitHub
#6101 (comment)
.

[aflaxman]

So cool, thanks all!

[deeptipatil]

I am still getting error on GradientBoostingClassifier.predict that " A sparse matrix was passed, but dense data is required.) Works fine with GradientBoostingClassifier.fit

[jnothman]

Please provide a runnable code snippet that reproduces the issue, mentioning the scikit-learn version