oob_score_[i] includes previous trees that were trained on the OOB instances of the i-th sample.
currently y_pred is updated for each sample (in and out bag) at each iteration. This way the OOB estimates are overly optimistic. When using OOB, we might use another y_pred (say y_oob_pred) that only contains the cumulative scores of those trees where the i-th sample was out-of-bag.
I've check gbm, it seems that they use the same implementation that we use ATM - I'll check the difference between the OOB scores on some toy examples...
This issue was brought up on the ML by Yanir .
currently
y_predis updated for each sample (in and out bag) at each iteration. This way the OOB estimates are overly optimistic. When using OOB, we might use anothery_pred(sayy_oob_pred) that only contains the cumulative scores of those trees where the i-th sample was out-of-bag.I've check gbm, it seems that they use the same implementation that we use ATM - I'll check the difference between the OOB scores on some toy examples...