Skip to content

feat: update following the metric change in substratools #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Esadruhn merged 4 commits into from
Sep 23, 2022
Merged

feat: update following the metric change in substratools #24

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a9d1f16
feat: update following the metric change in substratools
Esadruhn Sep 21, 2022
704f4ad
use dedicated docker image -temporary
Esadruhn Sep 22, 2022
c4ca833
changelog
Esadruhn Sep 22, 2022
d9d0467
revert before merge
Esadruhn Sep 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- Update the Client, it takes a backend type instead of debug=True + env variable to set the spawner - (#210)
- Do not use Model.category since this field is being removed from the SDK
- Update the tests and benchmark with the change on Metrics from substratools (#24)
Copy link
Copy Markdown
Contributor

@louishulot louishulot Sep 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has been decided with Romain that all notable changes within the whole API should be mention here under a
Notable Changes due to substra-tools title


### Changed

Expand Down Expand Up @@ -260,7 +261,7 @@ import substratools as tools
from sklearn.metrics import roc_auc_score


class AUC(tools.Metrics):
class AUC(tools.MetricAlgo):
def score(self, y_true, y_pred):
"""AUC"""
metric = roc_auc_score(y_true, y_pred) if len(set(y_true)) > 1 else 0
Expand All @@ -269,7 +270,7 @@ class AUC(tools.Metrics):


if __name__ == "__main__":
tools.metrics.execute(AUC())
tools.algo.execute(AUC())
```

the metric files should look like:
Expand All @@ -280,7 +281,7 @@ import substratools as tools
from sklearn.metrics import roc_auc_score


class AUC(tools.Metrics):
class AUC(tools.MetricAlgo):
def score(self, inputs, outputs):
"""AUC"""

Expand All @@ -293,7 +294,7 @@ class AUC(tools.Metrics):


if __name__ == "__main__":
tools.metrics.execute(AUC())
tools.algo.execute(AUC())
```

## 0.26.0 - 2022-08-22
Expand Down
4 changes: 2 additions & 2 deletions benchmark/camelyon/pure_substrafl/assets/metric.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from sklearn.metrics import roc_auc_score


class AUC(tools.Metrics):
class AUC(tools.MetricAlgo):
def score(self, inputs, outputs, task_properties):
"""AUC"""

Expand All @@ -19,4 +19,4 @@ def get_predictions(self, path):


if __name__ == "__main__":
tools.metrics.execute(AUC())
tools.algo.execute(AUC())
5 changes: 3 additions & 2 deletions tests/assets_factory.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@
import substratools as tools
import math
import numpy as np
class AccuracyMetric(tools.Metrics):

class AccuracyMetric(tools.MetricAlgo):
def score(self, inputs, outputs, task_properties):
# Datasamples are passed as a tuple of two elements: x and y
y_true = inputs['{InputIdentifiers.datasamples}'][1]
Expand All @@ -46,7 +47,7 @@ def load_predictions(self, path):


if __name__ == "__main__":
tools.metrics.execute(AccuracyMetric())
tools.algo.execute(AccuracyMetric())
"""

DEFAULT_OPENER_FILE = """
Expand Down