[MRG] accelerate stacking-classifier in test_common.py::test_ensemble_heterogeneous_estimators_behavior

[chritter]

Reference Issues/PRs

Towards #21407

What does this implement/fix? Explain your changes.

These changes are accelerating test case test_common.py::test_ensemble_heterogeneous_estimators_behavior

Any other comments?

#DataUmbrella Sprint

[ogrisel]

Thanks! I think it would be possible to make it run even faster with:

Suggested change

	("rf", RandomForestClassifier()),
	("rf", RandomForestClassifier(n_estimators=5, max_depth=3)),

Could you also update the related parametrized config for the same test function below?

Please also report the timings you get when running this test on your local machine with --durations=10.

[chritter]

@ogrisel Thank you very much for your suggestions. Note that this PR is still WIP and I plan to finish the optimization with your advice and the appropriate measurements reported.

[chritter]

Speed improvements reported as test duration 5-times averaged:

Setup	Test Duration
Original	2.52s
with cv=2	1.98s
with cv=2 + Randomforestclassifier(n_estimators=5, max_depth=3)	0.24s

Note: With final improvement LR fit: 27ms, RF fit: 17ms, , SVM fit: 1ms.

[chritter]

Based on @ogrisel suggestion I have committed the speed optimization of RandomForestRegressor(n_estimators=5, max_depth=3) for voting-classifier, stacking-regressor and voting-regressor. The following speed-ups are achieved (5-times measurements). To speed-up stacking-regressor I have also used cv=2.

Algo	Setup	Test Duration
Voting Classsifer	Original	0.60s
Voting Classsifer	Optimized	0.05s
StackingRegressor	Original	0.41s
StackingRegressor	Optimized	0.10s
VotingRegressor	Original	0.46s
VotingRegressor	Optimized	0.04s

[thomasjpfan]

Thank you for the PR @chritter !

LGTM!

[ogrisel]

Thanks very much! LGTM.