-
-
Notifications
You must be signed in to change notification settings - Fork 25.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add more D2 scores #20943
Comments
@lorentzenchr I'm thinking about taking a stab at this. I have a two questions about Q1: I can't seem to locate literature that specifically uses Q2, on file structure: Given that Tweedie applies to regression models and |
Just realized that it's the same as McFadden's R^2 which was much easier to search than e.g.
I think I'll take on |
@changhsinlee A McFadden R2 version for log loss is not controversial, IMO, so go ahead. A PR is most welcome. |
@lorentzenchr I started working on implementing |
Working on |
Hi @MaxwellLZH |
Hi @OmarManzoor, I'm not working on it at the moment, you can go ahead :) |
Thank you for the clarification. |
Is the |
@OmarManzoor No, |
I see. Could you kindly provide some reference to the specification of log_loss so that I can get some guidance on how to implement it? We have a log_loss defined in classification but this should be a bit different than that? |
I would still add a D2 Brier score and then we can finally close here. |
Since the approach for this seems similar, would it be okay if I work on this too? |
Describe the workflow you want to enable
I'd like to evaluate models by "percentage of reduction in metric X", where X is:
As
d2_tweedie_score
#17036, these are generalizations of ther2_score
.Describe your proposed solution
Add
d2_log_loss_score
d2_absolute_error_score
d2_pinball_loss_score
Describe alternatives you've considered, if relevant
No response
Additional context
See #17036 (comment).
The text was updated successfully, but these errors were encountered: