Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SensitivityAtSpecificity metric #2217

Merged
merged 69 commits into from
Feb 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
ee58904
fix typo of specicity
oguz-hanoglu Nov 2, 2023
54307c2
__init__.py's organized
oguz-hanoglu Nov 2, 2023
f3e38f2
added sensitivity_specificity metric
oguz-hanoglu Nov 2, 2023
1c69dad
update docstrings - add init.py entries
oguz-hanoglu Nov 2, 2023
c2d6aca
Merge branch 'master' into master
Borda Nov 10, 2023
584f351
Merge remote-tracking branch 'origin/master' into sensitivity_specifi…
oguz-hanoglu Nov 16, 2023
8c5ab9f
Merge branch 'master' into sensitivity_specificity
oguz-hanoglu Nov 16, 2023
1224bf0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 16, 2023
bd7f615
Merge branch 'master' into sensitivity_specificity
SkafteNicki Nov 26, 2023
896b2bd
changelog
SkafteNicki Nov 26, 2023
608f6b2
add rst doc
SkafteNicki Nov 26, 2023
207a0fd
fix init files
SkafteNicki Nov 28, 2023
83e0da4
add tests for plotting
SkafteNicki Nov 28, 2023
d520659
Merge branch 'master' into sensitivity_specificity
SkafteNicki Nov 28, 2023
2882b9f
Merge branch 'master' into sensitivity_specificity
SkafteNicki Nov 28, 2023
473310b
fix for inf threshold
SkafteNicki Nov 29, 2023
1434d79
add epsilon due to difference in tie breaking
SkafteNicki Nov 29, 2023
fa93d95
Merge branch 'master' into sensitivity_specificity
SkafteNicki Nov 29, 2023
2479f21
fixes for plot testing
SkafteNicki Nov 29, 2023
a049a52
move testing around
SkafteNicki Nov 29, 2023
e5d40b2
remove
SkafteNicki Nov 30, 2023
932d568
Merge branch 'master' into sensitivity_specificity
SkafteNicki Nov 30, 2023
c4a6c0c
Merge branch 'master' into sensitivity_specificity
mergify[bot] Dec 1, 2023
1a66a49
Apply suggestions from code review
Borda Dec 4, 2023
e9a153b
Apply suggestions from code review
Borda Dec 4, 2023
2163b17
Merge branch 'master' into sensitivity_specificity
Borda Dec 4, 2023
eb2dd25
Merge branch 'master' into sensitivity_specificity
mergify[bot] Dec 19, 2023
7dda946
Merge branch 'master' into sensitivity_specificity
mergify[bot] Dec 20, 2023
703fc7e
Merge branch 'master' into sensitivity_specificity
mergify[bot] Dec 20, 2023
1599300
Merge branch 'master' into sensitivity_specificity
mergify[bot] Dec 20, 2023
a7b724f
Merge branch 'master' into sensitivity_specificity
mergify[bot] Dec 20, 2023
f3ba45b
Merge branch 'master' into sensitivity_specificity
mergify[bot] Dec 20, 2023
b1b0b65
Merge branch 'master' into sensitivity_specificity
mergify[bot] Dec 20, 2023
e930770
Merge branch 'master' into sensitivity_specificity
mergify[bot] Dec 21, 2023
d5300db
Merge branch 'master' into sensitivity_specificity
Borda Dec 21, 2023
533a808
Merge branch 'master' into sensitivity_specificity
SkafteNicki Dec 22, 2023
8c0d5df
Merge branch 'master' into sensitivity_specificity
SkafteNicki Dec 22, 2023
44135fe
Merge branch 'master' into sensitivity_specificity
mergify[bot] Dec 22, 2023
f2f2317
Merge branch 'master' into sensitivity_specificity
Borda Dec 26, 2023
322c975
Merge branch 'master' into sensitivity_specificity
Borda Dec 27, 2023
7af1ade
Merge branch 'master' into sensitivity_specificity
mergify[bot] Dec 30, 2023
4bf2861
Merge branch 'master' into sensitivity_specificity
Borda Jan 4, 2024
c2fc89e
Merge branch 'master' into sensitivity_specificity
mergify[bot] Jan 9, 2024
0feacf7
Merge branch 'master' into sensitivity_specificity
mergify[bot] Jan 9, 2024
1200a67
Merge branch 'master' into sensitivity_specificity
Borda Jan 12, 2024
2652531
Merge branch 'master' into sensitivity_specificity
SkafteNicki Jan 29, 2024
b33f310
fix ddp testing after refactor
SkafteNicki Jan 29, 2024
67b7d52
Merge branch 'master' into sensitivity_specificity
mergify[bot] Jan 29, 2024
d4ece7d
Merge branch 'master' into sensitivity_specificity
mergify[bot] Jan 30, 2024
345abca
Merge branch 'master' into sensitivity_specificity
Borda Feb 6, 2024
133c727
Merge branch 'master' into sensitivity_specificity
Borda Feb 6, 2024
db3447d
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 6, 2024
b0eb876
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 6, 2024
5455588
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 6, 2024
7095ea8
Merge branch 'master' into sensitivity_specificity
Borda Feb 6, 2024
82c50d4
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 6, 2024
0f7f4f9
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 7, 2024
47e0c16
Merge branch 'master' into sensitivity_specificity
Borda Feb 7, 2024
cca3e7f
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 7, 2024
a12ae19
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 7, 2024
0ec7909
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 7, 2024
9ce1aee
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 8, 2024
77ea108
Merge branch 'master' into sensitivity_specificity
SkafteNicki Feb 13, 2024
6022154
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 13, 2024
8ccb7b2
Merge branch 'master' into sensitivity_specificity
Borda Feb 13, 2024
33f8f5a
skip on old pt versions
SkafteNicki Feb 14, 2024
3965c08
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 14, 2024
a266473
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 14, 2024
79ff057
Merge branch 'master' into sensitivity_specificity
mergify[bot] Feb 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

-
- Added `SensitivityAtSpecificity` metric to classification subpackage ([#2217](https://github.com/Lightning-AI/torchmetrics/pull/2217))


- Added `QualityWithNoReference` metric ([#2288](https://github.com/Lightning-AI/torchmetrics/pull/2288))
Expand Down Expand Up @@ -67,6 +67,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Added `aggregate` argument to retrieval metrics ([#2220](https://github.com/Lightning-AI/torchmetrics/pull/2220))
- Added utility functions in `segmentation.utils` for future segmentation metrics ([#2105](https://github.com/Lightning-AI/torchmetrics/pull/2105))


### Changed

- Changed minimum supported Pytorch version from 1.8 to 1.10 ([#2145](https://github.com/Lightning-AI/torchmetrics/pull/2145))
Expand Down
55 changes: 55 additions & 0 deletions docs/source/classification/sensitivity_at_specificity.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
.. customcarditem::
:header: Sensitivity At Specificity
:image: https://pl-flash-data.s3.amazonaws.com/assets/thumbnails/tabular_classification.svg
:tags: Classification

.. include:: ../links.rst

##########################
Sensitivity At Specificity
##########################

Module Interface
________________

.. autoclass:: torchmetrics.SensitivityAtSpecificity
:exclude-members: update, compute
:special-members: __new__

BinarySensitivityAtSpecificity
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. autoclass:: torchmetrics.classification.BinarySensitivityAtSpecificity
:exclude-members: update, compute

MulticlassSensitivityAtSpecificity
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. autoclass:: torchmetrics.classification.MulticlassSensitivityAtSpecificity
:exclude-members: update, compute

MultilabelSensitivityAtSpecificity
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. autoclass:: torchmetrics.classification.MultilabelSensitivityAtSpecificity
:exclude-members: update, compute

Functional Interface
____________________

.. autofunction:: torchmetrics.functional.classification.sensitivity_at_specificity

binary_sensitivity_at_specificity
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. autofunction:: torchmetrics.functional.classification.binary_sensitivity_at_specificity

multiclass_sensitivity_at_specificity
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. autofunction:: torchmetrics.functional.classification.multiclass_sensitivity_at_specificity

multilabel_sensitivity_at_specificity
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. autofunction:: torchmetrics.functional.classification.multilabel_sensitivity_at_specificity
2 changes: 2 additions & 0 deletions src/torchmetrics/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@
PrecisionRecallCurve,
Recall,
RecallAtFixedPrecision,
SensitivityAtSpecificity,
Specificity,
SpecificityAtSensitivity,
StatScores,
Expand Down Expand Up @@ -235,6 +236,7 @@
"SpearmanCorrCoef",
"Specificity",
"SpecificityAtSensitivity",
"SensitivityAtSpecificity",
"SpectralAngleMapper",
"SpectralDistortionIndex",
"SQuAD",
Expand Down
10 changes: 10 additions & 0 deletions src/torchmetrics/classification/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,12 @@
RecallAtFixedPrecision,
)
from torchmetrics.classification.roc import ROC, BinaryROC, MulticlassROC, MultilabelROC
from torchmetrics.classification.sensitivity_specificity import (
BinarySensitivityAtSpecificity,
MulticlassSensitivityAtSpecificity,
MultilabelSensitivityAtSpecificity,
SensitivityAtSpecificity,
)
from torchmetrics.classification.specificity import (
BinarySpecificity,
MulticlassSpecificity,
Expand Down Expand Up @@ -207,4 +213,8 @@
"BinaryPrecisionAtFixedRecall",
"MulticlassPrecisionAtFixedRecall",
"MultilabelPrecisionAtFixedRecall",
"BinarySensitivityAtSpecificity",
"MulticlassSensitivityAtSpecificity",
"MultilabelSensitivityAtSpecificity",
"SensitivityAtSpecificity",
]
Loading
Loading