Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] Rework of base series annotator API #6265

Merged
merged 65 commits into from
Jun 1, 2024
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
cbb611f
Add functions to check for valid tasks and learning types
Alex-JG3 Apr 4, 2024
5eb5c5c
Merge branch 'sktime:main' into refactor_base_series_annotator
Alex-JG3 Apr 5, 2024
54f3087
Remove `fmt` and `dense` and add `learning_type` and `task`
Alex-JG3 Apr 6, 2024
0290a87
Remove `fmt` and `labels` from PyOD adapter
Alex-JG3 Apr 6, 2024
26c064c
Add `task` and `learning_type` to `__init__`
Alex-JG3 Apr 6, 2024
dc7095f
Remove `fmt` and `labels` and add `task` and `learning_type`
Alex-JG3 Apr 6, 2024
a4cbb45
Change `task` and `learning_type` for the HMM classes
Alex-JG3 Apr 6, 2024
0f750e2
Remove `fmt` and `labels`, add `task` and `learning_type`
Alex-JG3 Apr 6, 2024
e7895a7
moved to tag
fkiraly Apr 6, 2024
737cf3d
Fix whitespace
Alex-JG3 Apr 10, 2024
48f3c55
Convert PyOD to tags
Alex-JG3 Apr 10, 2024
b960cdc
Convert to tags
Alex-JG3 Apr 10, 2024
d9e0908
Start a function to convert from sparse to dense
Alex-JG3 Apr 11, 2024
9b0e9ca
Initialise a default transform method
Alex-JG3 Apr 11, 2024
9920265
Update docstring example
Alex-JG3 Apr 11, 2024
26795a3
Add method for converting from dense to sparse formats
Alex-JG3 Apr 13, 2024
25d40d7
Update docstring examples
Alex-JG3 Apr 13, 2024
fe0ab78
Fix typos
Alex-JG3 Apr 13, 2024
951552b
Add method for converting change points to segments
Alex-JG3 Apr 13, 2024
286aa8f
Add function for converting change points to segments
Alex-JG3 Apr 13, 2024
835cd06
Add methods for predicting segments and change points
Alex-JG3 Apr 13, 2024
d4281f2
Change `transform` to use `predict_points` and `predict_segments`
Alex-JG3 Apr 13, 2024
a813709
Add length argument to sparse_to_dense and handle segmentation case
Alex-JG3 Apr 24, 2024
0cda3b9
Add test to check the annotator tags
Alex-JG3 Apr 25, 2024
48d46f6
Use public `predict_segments` instead of the private version
Alex-JG3 Apr 25, 2024
f200416
Change to use dataframes and series rather than numpy arrays
Alex-JG3 Apr 25, 2024
e7a7486
Convert pd.Series to array to avoid index issues
Alex-JG3 Apr 25, 2024
364d735
Add test sparse_to_dense function
Alex-JG3 Apr 27, 2024
c90e67a
Handle segments that do not cover the whole timeseries
Alex-JG3 Apr 28, 2024
1893d1a
Add more tests
Alex-JG3 Apr 28, 2024
c0bd630
Fix failing docstring examples
Alex-JG3 Apr 30, 2024
f4e7b37
Add length to the transform method
Alex-JG3 Apr 30, 2024
0e95960
Remove length from the transform arguments
Alex-JG3 Apr 30, 2024
f497345
Refactor ClaSPSegmentation to use new BaseSeriesAnnotator features
Alex-JG3 Apr 30, 2024
4da31ef
Update docstring and comments for sparse_to_dense
Alex-JG3 May 8, 2024
ba3fba2
Use pandas testing to assert frames and series
Alex-JG3 May 8, 2024
6a7fa08
Do not check dtypes when asserting frames and series
Alex-JG3 May 9, 2024
0cca27a
Add initial sparse to dense function using the interval datatype
Alex-JG3 May 11, 2024
e5ebae8
Add function to convert sparse points to dense format
Alex-JG3 May 11, 2024
fdf3e16
Change change_points_to_segments to use intervals
Alex-JG3 May 12, 2024
cafe3f3
Change segments_to_change_points to use intervals
Alex-JG3 May 12, 2024
4ca5232
Change the dense_to_sparse function to use intervals
Alex-JG3 May 12, 2024
401f6d7
Change sparse_to_dense to use intervals
Alex-JG3 May 14, 2024
5e5d460
Change how segments end points are found
Alex-JG3 May 18, 2024
df9f4ae
Change single ` to double `` for proper doc rendering
Alex-JG3 May 18, 2024
9860d87
Update docstrings
Alex-JG3 May 18, 2024
b0abeb7
Fix line that is too long
Alex-JG3 May 18, 2024
2f7cfc1
Merge branch 'main' into pr/6265
fkiraly May 19, 2024
b8f8ee6
Merge branch 'refactor_base_series_annotator' of https://github.com/A…
fkiraly May 19, 2024
d8c865d
Add fmt argument back in for backward compatibility
Alex-JG3 May 20, 2024
e86d7c8
Add deprecation warning.
Alex-JG3 May 20, 2024
fe617bd
Add the fmt and labels arguments back in for backwards compatibility
Alex-JG3 May 20, 2024
d319798
Remove DeprecationWarning import
Alex-JG3 May 20, 2024
4ce5d5e
Add fmt back to _predict_scores
Alex-JG3 May 20, 2024
c519a2e
Rewrite deprecation warning for fmt argument
Alex-JG3 May 21, 2024
43f22a8
Fix formatting
Alex-JG3 May 21, 2024
ada50a9
clasp warn
fkiraly May 22, 2024
bf2feda
pyod
fkiraly May 22, 2024
379e077
Update _pyod.py
fkiraly May 22, 2024
030678d
Merge branch 'main' into pr/6265
fkiraly May 22, 2024
66f2bab
linting
fkiraly May 22, 2024
fdaf7af
Update base.py
fkiraly May 22, 2024
6cd11b3
Revert "Update base.py"
fkiraly May 22, 2024
a0d0a67
Merge branch 'main' into pr/6265
fkiraly Jun 1, 2024
c61fbbd
Merge branch 'main' into pr/6265
fkiraly Jun 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
21 changes: 7 additions & 14 deletions extension_templates/annotation.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,16 +42,6 @@ class MySeriesAnnotator(BaseSeriesAnnotator):

Parameters
----------
fmt : str {"dense", "sparse"}, optional (default="dense")
Annotation output format:
* If "sparse", a sub-series of labels for only the outliers in X is returned,
* If "dense", a series of labels for all values in X is returned.
labels : str {"indicator", "score"}, optional (default="indicator")
Annotation output labels:
* If "indicator", returned values are boolean, indicating whether a value is an
outlier,
* If "score", returned values are floats, giving the outlier score.

parama : int
descriptive explanation of parama
paramb : string, optional (default='default')
Expand All @@ -69,6 +59,12 @@ class MySeriesAnnotator(BaseSeriesAnnotator):
and so on
"""

# Change the `task` and `learning_type` as needed
_tags = {
"task": "segmentation",
"learning_type": "unsupervised",
}

# todo: add any hyper-parameters and components to constructor
def __init__(
self,
Expand All @@ -77,8 +73,6 @@ def __init__(
est2=None,
paramb="default",
paramc=None,
fmt="dense",
labels="indicator",
):
# estimators should precede parameters
# if estimators have default values, set None and initialize below
Expand All @@ -89,8 +83,7 @@ def __init__(
self.paramb = paramb
self.paramc = paramc

# leave this as is
super().__init__(fmt=fmt, labels=labels)
super().__init__()

# todo: optional, parameter checking logic (if applicable) should happen here
# if writes derived values to self, should *not* overwrite self.parama etc
Expand Down
36 changes: 10 additions & 26 deletions sktime/annotation/adapters/_pyod.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,22 +21,17 @@ class PyODAnnotator(BaseSeriesAnnotator):
estimator : PyOD estimator
See ``https://pyod.readthedocs.io/en/latest/`` documentation for a detailed
description of all options.
fmt : str {"dense", "sparse"}, optional (default="dense")
Annotation output format:
* If "sparse", a sub-series of labels for only the outliers in X is returned,
* If "dense", a series of labels for all values in X is returned.
labels : str {"indicator", "score"}, optional (default="indicator")
Annotation output labels:
* If "indicator", returned values are boolean, indicating whether a value is an
outlier,
* If "score", returned values are floats, giving the outlier score.
"""

_tags = {"python_dependencies": "pyod"}
_tags = {
"python_dependencies": "pyod",
"task": "anomaly_detection",
"learning_type": "unsupervised",
}

def __init__(self, estimator, fmt="dense", labels="indicator"):
def __init__(self, estimator):
self.estimator = estimator # pyod estimator
super().__init__(fmt=fmt, labels=labels)
super().__init__()

def _fit(self, X, Y=None):
"""Fit to training data.
Expand Down Expand Up @@ -77,28 +72,17 @@ def _predict(self, X):
Returns
-------
Y : pd.Series - annotations for sequence X
exact format depends on annotation type
"""
fmt = self.fmt
labels = self.labels

X_np = X.to_numpy()

if len(X_np.shape) == 1:
X_np = X_np.reshape(-1, 1)

Y_np = self.estimator_.predict(X_np)

if labels == "score":
Y_val_np = self.estimator_.decision_function(X_np)
elif labels == "indicator":
Y_val_np = Y_np

if fmt == "dense":
Y = pd.Series(Y_val_np, index=X.index)
elif fmt == "sparse":
Y_loc = np.where(Y_np)
Y = pd.Series(Y_val_np[Y_loc], index=X.index[Y_loc])
# Convert the output to a sparse format
Y_loc = np.where(Y_np)
Y = pd.Series(Y_np[Y_loc], index=X.index[Y_loc])

return Y

Expand Down