Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix specificity calculation for CBPE #334

Merged
merged 6 commits into from
Nov 9, 2023
Merged

Conversation

nikml
Copy link
Contributor

@nikml nikml commented Nov 8, 2023

Calculation was grabbing wrong elements from the confusion matrix.

@nikml nikml added the bug Something isn't working label Nov 8, 2023
@nikml nikml self-assigned this Nov 8, 2023
@nikml nikml requested a review from nnansters as a code owner November 8, 2023 08:55
Copy link

codecov bot commented Nov 8, 2023

Codecov Report

Attention: 100 lines in your changes are missing coverage. Please review.

Comparison is base (dd20ef7) 83.40% compared to head (3232038) 82.55%.
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #334      +/-   ##
==========================================
- Coverage   83.40%   82.55%   -0.86%     
==========================================
  Files         100      100              
  Lines        7245     7550     +305     
  Branches     1275     1353      +78     
==========================================
+ Hits         6043     6233     +190     
- Misses        905      980      +75     
- Partials      297      337      +40     
Files Coverage Δ
...mance_calculation/metrics/binary_classification.py 85.47% <100.00%> (ø)
...annyml/sampling_error/multiclass_classification.py 100.00% <100.00%> (ø)
nannyml/thresholds.py 85.71% <ø> (ø)
...e_calculation/metrics/multiclass_classification.py 80.39% <84.61%> (+1.22%) ⬆️
...performance_estimation/confidence_based/metrics.py 85.96% <57.34%> (-9.88%) ⬇️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

cartgr and others added 5 commits November 9, 2023 20:24
* create tester

* Updated tester

* multiclass cm performance estimation

* Multiclass confusion matrix calc. and estimation + docs and tests for both

* Removed scratch testing files

* updating MCM docs

* Re-align docs with main version

* [skip ci] Update CHANGELOG.md

---------

Co-authored-by: Nikolaos Perrakis <nikos@nannyml.com>
Co-authored-by: Niels Nuyttens <niels@nannyml.com>
… them consistent with the dedicated realized performance calculator.
# Conflicts:
#	nannyml/performance_calculation/metrics/multiclass_classification.py
#	nannyml/performance_estimation/confidence_based/metrics.py
#	tests/performance_estimation/CBPE/test_cbpe_metrics.py
@nnansters nnansters merged commit 58d44bc into main Nov 9, 2023
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants