Hyperparameter tuning in River

[medha-chippa]

Hello, is there a way of being able to perform hyperparameter tuning in river. An equivalent of Grid search probably or any other function to perform hyperparameter tuning.

Thank you

[TxusLopez]

Check the "expert" module. It provides some ways of tuning for classification or regression tasks. For example, "SuccessiveHalvingClassifier" gives you the possibility of performing hyperparameter tuning through the Succesive Halving approach. If you are looking for RandomizedGridSearch, GridSearch or similar, to the best of my knowledge, they are not available.

[TxusLopez]

As you know (you saw my previous question on hyperparameter with sklearn techniques), there should be compatibility between sklearn tuning techniques (RandomizedGridSearch, GridSearch) and river estimators, but I have not been able to overcome some problems by now.

[TxusLopez]

River just considers a few tuning methods. Random Grid Search and Grid Search are still out of river. It is true that Sucessive Halving achieves a good equilibrium between time and goodness of solutions, but in some cases you may need an exahustive search to find the best configuration even consuming much time. And yes, you can use Sucessive Halving with trees and other classifiers.

[TxusLopez]

Using the "compat" module, we should be able to use other sklearn functionalities

[medha-chippa]

Thank you so much. Some of the models in river and their corresponding parameters are a little different from the ones in scikit learn. Is there some kind of documentation/ reference guide that I could refer to for building a parameter grid. The permissible ranges of values for the hyperparameters in river are not mentioned in the API reference, the default values for the same are only mentioned, with the values for only a few hyperparameters having strings as their values being mentioned like the criterion parameter in tree based methods.

[TxusLopez]

Bear in mind that most of sklearn techniques are not incremental, except those which implement the "partial_fit" method. River techniques are incremental, so that you can find some differences in some parameters (those which are specific for incremental learning). E.g. you cannot directly compare all the parameters of a (traditional) decission tree with a Hoeffding decision tree. The latter one implements mechanisms to grow and adapt incrementally. However, other parameters are shared (max depth, split criterion ...).

The main difference I find between sklearn and river e.g. with the "Succesive Halving" method, is that the river one does not perform cross-validation (I think), because I do not find the "cv" parameter in it. Is that correct? If yes, and if it has removed to be faster, how compensate that feature, e.g. to avoid overfitting?

[medha-chippa]

Yes the version on river doesn't perform cross validation. We would have to probably manually segment the data and test for over fitting. In everybody's best interests, it would be nice for us to be able to convert river models to scikit learn models using River2SKLClassifier or River2SKLRegressor.