Some questions about YDF

[lusis-ai]

Hi,

A few questions about YDF.

1. how the old out automatically extracted from the training set (in CART or RF) is built ? is it a random picking in the training samples ? or is it splitted by using only the very end of the training samples as validation set (so making it compatible with a hold out set for time series) ?

Example : if training set is [1,2,3,4,5,6,7,8,9,10] and I set validation_ratio=0.3 does the train/val are respectively : [1,2,3,4,5,6,7] and [8,9,10]?

3. in [s:0.42281 n:20 np:8 miss:0] : n is the number of samples, but what np means ? (here it's not the number of positive as I have 10 of each class in this example.

4. What is the score for CART or RF Classifier ? Gini ? Entropy ? Other ?

5. Is there a way to plot the trees like with sklearn (not the text output but the same as sklearn.tree.plot_tree()), so showing the same information (score, samples, feature condition, each class proba, ...) ? or is it planned in the future ? or maybe the simpler is to export model then reimport in tf-df and use DTreeViz ?

Thanks for your help

[rstz]

1. RF do not use a validation dataset (by definition of the random forest algorithm, see answer 2). For CART and GBT, the validation dataset is randomly chosen indices, it is not chosen from the end of the dataset. If you require this property (e.g. for forecasting), you must create the validation dataset yourself and provide it. Code for CART, GBT is similar.
2. np: is the number of positive examples. If this is a Random Forest model, note that the individual trees are not trained on the original dataset, but each one on a separate dataset sampled with replacement. The bootstrap_training_dataset hyperparameter controls this behaviour.
3. For categorical splits, YDF uses entropy as score. for numerical splits it uses variance, for categorical sets, the score is explained in this paper
4. TF-DF style plots will be there very soon (I think, no promises). We haven't thought about dtreeviz - if exporting to TF-DF and applying dtreeviz works, that sounds like a good workaround for the moment. If dtreeviz is useful to people, it should be very little work to add dtreeviz support for YDF as well and we'll put it on the roadmap.

[lusis-ai]

Hi,

Thanks.

np : ok it's the number of positive after the split of the displayed node, not in the current node (here the root) but on the "pos" child node ... it explains why I have 8 instead of 10.

[lusis-ai]

I've tried to test plots from TF-DF after import of YDF saved model. But when trying to call model.to_tensorflow_saved_model(), my Jupyterkernel crashed.

I've added the issue to the #79 previously created as it fall in the same category of other packages not working when imported with ydf.

[rstz]

Plotting with PYDF just landed on the main branch and will be included in the next release of the Python wrapper

[lusis-ai]

Great.
I pull and tried but I get an error after calling model.plot_tree() (I replaced my username by XXXXX)/

FileNotFoundError: [Errno 2] No such file or directory: /XXXXXXX/miniforge3/envs/airesearch/lib/python3.11/site-packages/ydf/model/tree/plotter.js'

Seems it's not in the wheel even if in the git.

I copied it as a workaround and it works.

[rstz]

Nice, thank you for trying it out - Manifest.in should be include ydf/model/tree/plotter.js instead of include model/tree/plotter.js and this was not yet caught by tests. I'll fix this before the 0.3.0 release