New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FIX make sure the decision function of weak learner is symmetric #26521
Conversation
Since this is a bug that I found while looking at #26486, maybe @adrinjalali and @StefanieSenger are interested to look at it. (My next step is to open an issue and ask to deprecate completely SAMME.R because it is actually an unpublished algorithm that triggers some divergent behaviour). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh my!
Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>
@ogrisel do you agree with the last changes ? |
Solved the conflicts |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. However I made a mistaken when resolving the changelog conflict. Let me push a fix before merging.
…kit-learn#26521) Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com> Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
…kit-learn#26521) Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com> Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
…kit-learn#26521) Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com> Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
closes #26520
From the original paper, when considering the multiclass exponential-loss, it is expected that the decision function of each weak-learner to be symmetric around zero.
It concretely means that the predicted class should be
(n_classes - 1) / n_classes * weight
and the other classes-1 / n_classes * weight
.This PR correct the
decision_function
and thestaged_decision_function
and add a non-regression case where we use a single learner to make sure that we respect this constraint.