Documentation mismatch about categorical variable handling in Decision Trees

[hiteshsom]

The below link specifies that the of decsion trees are

• Able to handle both numerical and categorical data. Other techniques are usually specialised in analysing datasets that have only one type of variable. See algorithms for more information.

But if you click on more in the above point it takes you to page
https://scikit-learn.org/stable/modules/tree.html#tree-algorithms

and there is a sentence saying: scikit-learn uses an optimised version of the CART algorithm; however, scikit-learn implementation does not support categorical variables for now.

[jnothman]

It supports them naively rather than in accordance with the published algorithms noted there. Suggested improvements to the language are welcome, ideally via a pull request.

[rth]

Resolved in #18106

It supports them naively rather than in accordance with the published
algorithms noted there.

Sorry @jnothman I didn't see this comment before merging. I would consider having to run OrdinalEncoding before as not supporting categorical variables, but I could be missing something. Would you have preferred to have that PR worded differently then?

[hiteshsom]

Sorry if my answer is silly @jnothman @rth

but even OrdinalEncoding will convert a category column in a numeric column. So will that oridnally encoded column be taken as a real categorical variable by Decision Tree. Also even if we convert the ordinally encoded column to pandas Category type but does Decison Tree accept such input or it internally converts all the columsn to numeric types ?

What if it is converted to one hot encoding before passing to the fit or fit_transform method ?

[hiteshsom]

Is it ok to convert categories using pd.get_dummies within Decision tree estimator's fit method? Other option may be to use OneHotEncoding ?

[glemaitre]

This is not a good strategy to use dummy or OHE with a decision tree since it will increase the number of features and make the algorithm much slower. OrdinalEncoder should be used instead.

[glemaitre]

but even OrdinalEncoding will convert a category column in a numeric column. So will that oridnally encoded column be taken as a real categorical variable by Decision Tree. Also even if we convert the ordinally encoded column to pandas Category type but does Decison Tree accept such input or it internally converts all the columsn to numeric types ?

The trees can easily isolate categories which are encoded numerically as well. This is one of the strategies on how to treat categorical data.

[hiteshsom]

but even OrdinalEncoding will convert a category column in a numeric column. So will that oridnally encoded column be taken as a real categorical variable by Decision Tree. Also even if we convert the ordinally encoded column to pandas Category type but does Decison Tree accept such input or it internally converts all the columsn to numeric types ?

The trees can easily isolate categories which are encoded numerically as well. This is one of the strategies on how to treat categorical data.

Ok. Sorry, I had no clue of that. I will read through code and/or online to understand on decision trees working better. But then it was already written in the documentation inside that current implementation of decision does not handle categorical data so did that refer to string type of datatypes ?

[glemaitre]

But then it was already written in the documentation inside that current implementation of decision does not handle categorical data so did that refer to string type of datatypes ?

Yes. Categories encoded with integer will be supported then. You can have a look at this notebook: https://github.com/lesteve/2020-scikit-learn-tutorial/blob/master/notebooks/trees.ipynb

This might be useful. I will close this issue since the documentation has been updated.