The 'predict' method of RandomForestRegressor runs too slowly when data is "Big"

[jdk8]

Please execuse my poor English.

My codes is like this:
clf = RandomForestRegressor(random_state=0, n_estimators=100,n_jobs=4)
traindataX=StandardScaler().fit_transform(traindataX)
clf=clf.fit(traindataX,traindatay)

traindataX has 16 features and 889054 samples, each feature's value of traindataX is 0~1 before StandardScaler().fit_transform.
My machine has 4GB RAM, intel core i3 2.53GHZ CPU. Win7 64bits.

After about 1 hour's training, I need to predict test samples. But the prediction for a single sample will take about 4 minutes, and the memory is full, the prediction time is too long for me.
Is there any way to optimize the prediction speed of RandomForestRegressor?

Thanks!

[chiragnagpal]

I think what is happening here is that your exhausting your RAM. And hence, some of the memory is being swapped out. Hence, during prediction you need to do multiple read write cycles from the Physical Disk, making it too slow. Here are some options to explore:
1)Reduce number of estimators
2)Reduce max_depth
3)Increase Min Split value

The above solutions come at a cost of accuracy.

The other option is, to free memory in your system for Python. An easy way to do this is to load your computer in a text only mode, without any display manager.

You can try other options too.

[jdk8]

Thanks Chirag! @chiragnagpal I will try your great suggestions, keep in touch!

[cheekybastard]

@jianpingjwang I'm having the same issue, it might be a bug associated with the parameter 'n_jobs=' in RandomForestRegressor. If you watch your CPU load levels with 'n_jobs=4' when training your model all CPU load levels 100%, yet with prediction all CPU load levels are < 15%. Very strange for all CPUs to be close to idling, and not max out with a prediction work load.

Try setting it as ''n_jobs=1", thats the only setting returning predictions in a reasonable amount of time (1 CPU 100%).

[jdk8]

@cheekybastard Thanks!
I also find the strange CPU performance. I ever used the setting ''n_jobs=1", but the amount of time didn't satisfy me, then I tried ''n_jobs=4", hoping it will reduce the time.

Have you ever tried the Random Forests library in OpenCV? Maybe it will predict fast. I will have a try.

[glouppe]

What version of scikit-learn are you currently using?

[cheekybastard]

I'm using version: '0.17.dev0'

[amueller]

you should probably use some regularization. You are building trees to full depth with 889054 samples. So the trees will likely be gigantic. I don't think it will come at a cost of predictive power, it will probably improve it.

[amueller]

can you maybe benchmark the prediction with n_jobs=1 and n_jobs=4?

[jdk8]

@glouppe I'm using scikit-learn 0.16.

[jdk8]

Please excuse me, I typed wrong, there are about 400000 samples.

[jdk8]

@amueller Thanks, I will try regularization. The prediction times of the two setting are almost the same.

[jdk8]

The random forest library of OpenCV provides an interactive prediction with the same dataset and parameters, much faster than scikit-learn.

[amueller]

really? That is pretty surprising. Can you give the parameters and timing results?

What do you mean by "the two settings"?

[jdk8]

@amueller
The parameter of OpenCV is like this:
CvRTParams(
5000,
1,
0,
false,
16,
0,
true,
4,
100,
0.001f,
CV_TERMCRIT_EPS
));

The scikit-learn's parameter's clf = RandomForestRegressor(random_state=0, n_estimators=100)
Later I will give the timing results.

"the two settings" means the setting of "n_jobs=1" and "n_jobs=4".

[glouppe]

Hi,

The comparison is not fair because OpenCV default hyper-parameters are
different from ours. In particular, they use stopping criteria different
from ours. For a fair comparison, you should set parameters as follows.
This would mimick what we do in RandomForestClassifier:

CvRTParams(
1000000, // <- this
1,
0,
false,
16,
0,
true,
sqrt(n_features), // <- this
100,
0.0f, // <- this
CV_TERMCRIT_ITER // <- this
));

Best,
Gilles

On 13 July 2015 at 13:12, Jianping notifications@github.com wrote:

@amueller https://github.com/amueller
The parameter of OpenCV is like this:
CvRTParams(
5000,
1,
0,
false,
16,
0,
true,
4,
100,
0.001f,
CV_TERMCRIT_EPS
));

The scikit-learn's parameter's clf = RandomForestRegressor(random_state=0,
n_estimators=100)
Later I will give the timing results.

"the two settings" means the setting of "n_jobs=1" and "n_jobs=4".

—
Reply to this email directly or view it on GitHub
#4935 (comment)
.