Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added regression_MWI_score function #408

Merged
merged 8 commits into from
Mar 1, 2024

Conversation

Carl-McBride-Ellis
Copy link
Contributor

@Carl-McBride-Ellis Carl-McBride-Ellis commented Jan 27, 2024

Description

This is an efficient implementation for calculating the mean Winkler interval score, based on the code written by Kaggle user AmbrosM and published in the topic "Accelerating the scoring function".

Fixes #407

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How Has This Been Tested?

Private testing was performed in the Kaglge competition Prediction interval competition I: Birth weight by comparing correspondence with the results to those output by the competition metric code published in Winkler Interval score metric.

Update (2024-02-27)

A test routine for the function regression_MWI_score has been written and added to the end of mapie/tests/test_metrics.py (b101cb0).

Checklist

  • I have read the contributing guidelines
  • I have updated the HISTORY.rst and AUTHORS.rst files
  • Linting passes successfully : make lint
  • Typing passes successfully : make type-check
  • Unit tests pass successfully : make tests
  • Coverage is 100% : make coverage
  • Documentation builds successfully : make doc

@LacombeLouis LacombeLouis self-requested a review January 29, 2024 08:26
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (614293e) to head (5ebeee1).
Report is 267 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##            master      #408    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           39        39            
  Lines         4616      4896   +280     
  Branches       487       801   +314     
==========================================
+ Hits          4616      4896   +280     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@LacombeLouis LacombeLouis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Carl-McBride-Ellis for this PR. We hope to soon make a release of this new metric so it can be used in the Kaggle competition!

Note that we will need to add this change to the history and authors file.

@LacombeLouis
Copy link
Collaborator

LacombeLouis commented Feb 28, 2024

We should definitely add some documentation, but we can do this in a separate PR, so this is pushed through as quickly as possible!

New issue: #415.

Copy link
Collaborator

@candicemyt candicemyt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Carl-McBride-Ellis for this PR! It is something that we have wanted to add for a long time.
I have just added a few comments in order to be consistent with the rest of the library

mapie/metrics.py Outdated Show resolved Hide resolved
mapie/metrics.py Outdated Show resolved Hide resolved
@LacombeLouis LacombeLouis merged commit 6dcb570 into scikit-learn-contrib:master Mar 1, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add the Winkler Interval score
4 participants