Skip to content

[BUG] Fix SquaredDistrLoss: use pdf instead of log_pdf in loss formula#937

Merged
fkiraly merged 2 commits into
sktime:mainfrom
MayankSharma-2812:fix/squared-distr-loss
Mar 17, 2026
Merged

[BUG] Fix SquaredDistrLoss: use pdf instead of log_pdf in loss formula#937
fkiraly merged 2 commits into
sktime:mainfrom
MayankSharma-2812:fix/squared-distr-loss

Conversation

@MayankSharma-2812
Copy link
Copy Markdown
Contributor

@MayankSharma-2812 MayankSharma-2812 commented Mar 16, 2026

Reference Issues/PRs

Fixes #934. Merged jointly with #935 for shared credit.

What does this implement/fix? Explain your changes.

SquaredDistrLoss._evaluate_by_index in skpro/metrics/_classes.py used log_pdf instead of pdf when computing the squared distribution loss.

The squared distribution loss (continuous Brier / Gneiting score) is defined as:

L(y, d) = -2 · p_d(y) + ‖p_d‖²

where p_d(y) is the probability density evaluated at the observation y.

The implementation previously used:

res = -2 * y_pred.log_pdf(y_true) + y_pred.pdfnorm(a=2)

This PR corrects the implementation to:

res = -2 * y_pred.pdf(y_true) + y_pred.pdfnorm(a=2)

Using log_pdf leads to incorrect loss values. For example, for a Normal(0,1) distribution evaluated at its mode (y = 0), the correct loss is approximately -0.52, while the previous implementation produced +2.12.

Does your contribution introduce a new dependency?

No.

What should a reviewer concentrate their feedback on?

  • The one-line fix in skpro/metrics/_classes.py.
  • The added unit tests verifying the corrected behaviour.

Did you add any tests for the change?

Yes. Two tests were added to skpro/metrics/tests/test_distr_metrics.py:

  • test_squared_distr_loss_uses_pdf – verifies that the loss uses pdf and not log_pdf.
  • test_squared_distr_loss_multivariate – confirms that multivariate output shape and column naming remain unchanged.

All existing tests pass locally.

PR checklist

  • The PR title starts with [BUG]
  • Tests added for the change
  • No new dependencies introduced

Copy link
Copy Markdown
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, but your PR contains a lot of other changes.

Can you please make sure that you only commit the fixes to the squared distribution loss?

@fkiraly fkiraly added enhancement module:metrics&benchmarking metrics and benchmarking modules labels Mar 16, 2026
Copy link
Copy Markdown
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

@fkiraly fkiraly merged commit c83950a into sktime:main Mar 17, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement module:metrics&benchmarking metrics and benchmarking modules

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] SquaredDistrLoss uses log_pdf instead of pdf

2 participants