[BUG] Fix SquaredDistrLoss: use pdf instead of log_pdf in loss formula#937
Merged
Merged
Conversation
fkiraly
requested changes
Mar 16, 2026
Collaborator
fkiraly
left a comment
There was a problem hiding this comment.
Good catch, but your PR contains a lot of other changes.
Can you please make sure that you only commit the fixes to the squared distribution loss?
b498812 to
67eafdb
Compare
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs
Fixes #934. Merged jointly with #935 for shared credit.
What does this implement/fix? Explain your changes.
SquaredDistrLoss._evaluate_by_indexinskpro/metrics/_classes.pyusedlog_pdfinstead ofpdfwhen computing the squared distribution loss.The squared distribution loss (continuous Brier / Gneiting score) is defined as:
L(y, d) = -2 · p_d(y) + ‖p_d‖²
where
p_d(y)is the probability density evaluated at the observationy.The implementation previously used:
This PR corrects the implementation to:
Using
log_pdfleads to incorrect loss values. For example, for aNormal(0,1)distribution evaluated at its mode (y = 0), the correct loss is approximately -0.52, while the previous implementation produced +2.12.Does your contribution introduce a new dependency?
No.
What should a reviewer concentrate their feedback on?
skpro/metrics/_classes.py.Did you add any tests for the change?
Yes. Two tests were added to
skpro/metrics/tests/test_distr_metrics.py:test_squared_distr_loss_uses_pdf– verifies that the loss usespdfand notlog_pdf.test_squared_distr_loss_multivariate– confirms that multivariate output shape and column naming remain unchanged.All existing tests pass locally.
PR checklist
[BUG]