Polynomial Detrending Component #1992

freddyaboulton · 2021-03-17T20:00:30Z

Pull Request Description

Fix #1944

After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of docs/source/release_notes.rst to include this pull request by adding :pr:123.

codecov · 2021-03-17T21:08:55Z

Codecov Report

Merging #1992 (e01ee05) into main (6eaa1dc) will decrease coverage by 0.1%.
The diff coverage is 98.0%.

@@            Coverage Diff            @@
##             main    #1992     +/-   ##
=========================================
- Coverage   100.0%   100.0%   -0.0%     
=========================================
  Files         276      278      +2     
  Lines       22571    22714    +143     
=========================================
+ Hits        22565    22705    +140     
- Misses          6        9      +3

Impacted Files	Coverage Δ
evalml/pipelines/components/__init__.py	`100.0% <ø> (ø)`
evalml/tests/utils_tests/test_dependencies.py	`85.8% <50.0%> (-14.2%)`	⬇️
evalml/pipelines/components/component_base_meta.py	`100.0% <100.0%> (ø)`
...alml/pipelines/components/transformers/__init__.py	`100.0% <100.0%> (ø)`
.../components/transformers/preprocessing/__init__.py	`100.0% <100.0%> (ø)`
...transformers/preprocessing/polynomial_detrender.py	`100.0% <100.0%> (ø)`
evalml/tests/component_tests/test_components.py	`100.0% <100.0%> (ø)`
...tests/component_tests/test_polynomial_detrender.py	`100.0% <100.0%> (ø)`
evalml/tests/component_tests/test_utils.py	`100.0% <100.0%> (ø)`
evalml/tests/conftest.py	`100.0% <100.0%> (ø)`
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6eaa1dc...e01ee05. Read the comment docs.

freddyaboulton · 2021-03-18T18:40:37Z

evalml/pipelines/components/component_base_meta.py

            klass = type(self).__name__
            if not self._is_fitted and self.needs_fitting:
                raise ComponentNotYetFittedError(f'This {klass} is not fitted yet. You must fit {klass} before calling {method.__name__}.')
+            elif method.__name__ == 'inverse_transform':


This is a short-term fix to make the component metaclass aware of the inverse_transform method. The current implementation will not work because if y is None, the method will run with only the X argument which is not what we want.

I think we can change this implementation to something like:

def _check_for_fit(self, *args, **kwargs): klass = type(self).__name__ if not self._is_fitted and self.needs_fitting: raise ComponentNotYetFittedError(f'This {klass} is not fitted yet. You must fit {klass} before calling {method.__name__}.') return method(self, *args, **kwargs)

I think this would be more maintainable in the long-term. I will file an issue for this!

This makes sense! Just for my own edification, why would this new line be necessary? Why not raise a ValueError in transform for when y is None?

@ParthivNaresh I think we're currently raising an exception in inverse_transform if y is None. The problem is that the meta class doesn't pass the y argument if it's None to inverse_transform in the current implementation. This causes an error because we're expecting a y argument, even if it's None.

freddyaboulton · 2021-03-19T17:01:48Z

evalml/pipelines/components/transformers/preprocessing/polynomial_detrender.py

+        y_series = _convert_woodwork_types_wrapper(y_dt.to_series())
+        y_t = self._component_obj.transform(y_series)
+        y_t = ww.DataColumn(pd.Series(y_t, index=y_series.index))
+        return X, y_t


I'm returning X and y because our ComponentGraph assumes targets are returned as the second value in a tuple

freddyaboulton · 2021-03-19T17:09:33Z

FYI it's impossible to get 100% coverage on the patch:

        # We don't expect to install sktime in python 3.9. Let's verify it's not present:
        if 'sktime' in module and is_running_py_39_or_above:
            with pytest.raises(ModuleNotFoundError):
                import_module(module)
            continue

Those lines only run when python 3.9 is installed and we don't measure coverage for python 3.9. But the fact that the 3.9 tests pass means that those lines are run 😄

ParthivNaresh

Everything looks great, nicely done!

ParthivNaresh · 2021-03-20T15:16:06Z

evalml/pipelines/components/transformers/preprocessing/polynomial_detrender.py

+                If 2, quadratic model is fit, etc.
+            random_seed (int): Seed for the random number generator. Defaults to 0.
+        """
+


Should we error check for degree if passed in as a float? Perhaps casting it to an int and then adding a test for it?

Great idea!

ParthivNaresh · 2021-03-20T15:17:09Z

evalml/pipelines/components/transformers/preprocessing/polynomial_detrender.py

+
+        Arguments:
+            X (ww.DataTable, pd.DataFrame, optional): Ignored.
+            y (ww.DataColumn, pd.Series): Target variable to detrend. Ignored.


Why is y ignored here?

Typo 😂 Good catch!

ParthivNaresh · 2021-03-20T19:28:58Z

evalml/tests/component_tests/test_components.py

            component_with_different_kwargs = component_class(diff_test_arg="test")
            assert component.parameters['test_arg'] == "test"
-            assert component._component_obj.test_arg == "test"
+            if not isinstance(component, PolynomialDetrender):


Why is PolynomialDetrender left out of this?

Raises an sklearn exception!

ParthivNaresh · 2021-03-20T19:44:32Z

evalml/pipelines/components/component_base_meta.py

            klass = type(self).__name__
            if not self._is_fitted and self.needs_fitting:
                raise ComponentNotYetFittedError(f'This {klass} is not fitted yet. You must fit {klass} before calling {method.__name__}.')
+            elif method.__name__ == 'inverse_transform':


This makes sense! Just for my own edification, why would this new line be necessary? Why not raise a ValueError in transform for when y is None?

ParthivNaresh · 2021-03-20T20:20:49Z

evalml/tests/component_tests/test_polynomial_detrender.py

+
+@pytest.mark.parametrize("input_type", ['np', 'pd', 'ww'])
+@pytest.mark.parametrize("use_int_index", [True, False])
+@pytest.mark.parametrize("degree", [1, 2, 3])


bchen1116

Looks great to me! The test coverage is solid.

angela97lin

LGTM, pretty darn cool stuff--the tests were great in understanding what's going on. Left some nitpicky comments per usual but 👍👍👍

angela97lin · 2021-03-22T18:48:53Z

docs/source/install.ipynb

+    "\n",
+    "## Python 3.9 support\n",
+    "\n",
+    "Evalml can still be installed with pip in python 3.9 but note that `sktime`, one of our dependencies, will not be installed because that library does not yet support python 3.9. This means the ``PolynomialDetrending`` component will not be usable in python 3.9. You can try to install `sktime` [from source](https://www.sktime.org/en/latest/installation.html#building-from-source) in python 3.9 to use the ``PolynomialDetrending`` component but be warned that we only test it in python 3.7 and 3.8."


evalml/pipelines/components/transformers/preprocessing/polynomial_detrender.py

angela97lin · 2021-03-22T18:56:57Z

evalml/tests/component_tests/test_polynomial_detrender.py

+def test_polynomial_detrender_raises_value_error_target_is_none(ts_data):
+    X, y = ts_data
+
+    with pytest.raises(ValueError, match="y cannot be None for PolynomialDetrender!"):


Realllly nit-picking now but do we want to add tests for fit/transform too for completeness :')

angela97lin · 2021-03-22T18:58:04Z

evalml/tests/component_tests/test_polynomial_detrender.py

+    if input_type == 'np':
+        X = X_input.values
+        y = y_input.values
+    elif input_type == 'ww':
+        X = ww.DataTable(X_input)
+        y = ww.DataColumn(y_input)


We can use the make_data_type fixture for this I believe? hehe

Thanks for the tip! I didn't know about make_data_type! I'm not sure if it's worth it in this case because there is no pandas to numpy conversion and the woodwork conversion is straight-forward enough in this case. But I will keep it in mind in the future.

dsherry · 2021-03-23T20:39:23Z

requirements.txt

 category_encoders>=2.0.0
 statsmodels >= 0.12.2
 imbalanced-learn>=0.7.0
+sktime>=0.5.3;python_version<"3.9"


👍 yep good call putting this in requirements and note core-requirements, lgtm

dsherry · 2021-03-23T20:42:29Z

evalml/tests/conftest.py


+@pytest.fixture
+def is_running_py_39_or_above():
+    return sys.version_info >= (3, 9)


dsherry · 2021-03-23T20:43:41Z

evalml/tests/utils_tests/test_dependencies.py

+        if module == 'sktime' and is_running_py_39_or_above:
+            with pytest.raises(ModuleNotFoundError):
+                import_module(module)
+            continue


Got it. So the reason this doesn't appear in our coverage report is because we're currently only generating coverage on python 3.8. 👍

dsherry · 2021-03-23T20:45:11Z

evalml/pipelines/components/transformers/preprocessing/polynomial_detrender.py

+        detrender = detrend.Detrender(trend.PolynomialTrendForecaster(degree=degree))
+
+        super().__init__(parameters=params,
+                         component_obj=detrender,


Is there a reason we need to set component_obj for this component? It appears you're doing custom logic for fit/transform/fit_transform, and that was the main utility of passing component_obj down to the base class

dsherry

🚢 !

freddyaboulton force-pushed the 1944-polynomial-detrending-component branch 2 times, most recently from b83517e to 5a6083b Compare March 18, 2021 18:35

freddyaboulton commented Mar 18, 2021

View reviewed changes

freddyaboulton force-pushed the 1944-polynomial-detrending-component branch from 092a02e to f27b798 Compare March 19, 2021 14:23

freddyaboulton marked this pull request as ready for review March 19, 2021 16:59

auto-assign bot assigned freddyaboulton Mar 19, 2021

freddyaboulton requested a review from dsherry March 19, 2021 17:00

freddyaboulton commented Mar 19, 2021

View reviewed changes

freddyaboulton requested review from ParthivNaresh, angela97lin, bchen1116, chukarsten and jeremyliweishih March 19, 2021 17:01

freddyaboulton force-pushed the 1944-polynomial-detrending-component branch from f27b798 to 29b973d Compare March 19, 2021 19:17

ParthivNaresh approved these changes Mar 20, 2021

View reviewed changes

bchen1116 approved these changes Mar 22, 2021

View reviewed changes

angela97lin approved these changes Mar 22, 2021

View reviewed changes

freddyaboulton added 11 commits March 23, 2021 11:13

Adding PolynomialDetrender and unit tests.

750e9da

Linting.

7abb1b8

Adding polynomial detrending source.

43053a6

Adding PR 1992 to release notes.

9824905

Fixing tests and adding coverage for np and ww.

4f7f7a7

Adding sktime to latest dependency versions.

d68bc9b

Fixing \ in grep for lint.

923014b

ComponentBaseMeta checks fit before running inverse transform.

6753030

Restricting sktime to python < 3.9

f1b7b92

Fixing 3.9 tests to account for the fact sktime won't be installed.

25d1cd6

Add to api ref.

7ed0332

Type checking degree parameter.

b9e825d

freddyaboulton force-pushed the 1944-polynomial-detrending-component branch from 29b973d to b9e825d Compare March 23, 2021 15:15

Fix len(all_components()) test.

e01ee05

dsherry reviewed Mar 23, 2021

View reviewed changes

evalml/tests/conftest.py

@pytest.fixture

def is_running_py_39_or_above():

return sys.version_info >= (3, 9)

Copy link

Contributor

dsherry Mar 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nifty!

dsherry reviewed Mar 23, 2021

View reviewed changes

dsherry approved these changes Mar 23, 2021

View reviewed changes

dsherry merged commit a1d04e0 into main Mar 23, 2021

dsherry mentioned this pull request Mar 24, 2021

Release v0.21.0 #2029

Merged

freddyaboulton deleted the 1944-polynomial-detrending-component branch June 3, 2021 14:02

Polynomial Detrending Component #1992

Polynomial Detrending Component #1992

Uh oh!

Conversation

freddyaboulton commented Mar 17, 2021

Pull Request Description

Uh oh!

codecov bot commented Mar 17, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

freddyaboulton Mar 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

freddyaboulton commented Mar 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ParthivNaresh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bchen1116 left a comment

Choose a reason for hiding this comment

Uh oh!

angela97lin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsherry left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov bot commented Mar 17, 2021 •

edited

Loading

freddyaboulton Mar 18, 2021 •

edited

Loading

freddyaboulton commented Mar 19, 2021 •

edited

Loading