Component parameters: save all fields for builtin components #847

dsherry · 2020-06-12T00:52:18Z

Changes

Define a ComponentBase.parameters getter, to keep the saved parameters immutable after component init
Make the args in ComponentBase.__init__ optional with defaults: parameters (empty dict), component_obj (None) and random_state (0).
Update all our components to save all input values in parameters (except for random_state)
Add test coverage to enforce the last point for all components listed in all_components
Exclude ComponentBase, Transformer, Estimator and other transformer base classes from all_components list

… init

…mators

dsherry · 2020-06-12T00:58:47Z

evalml/pipelines/components/transformers/drop_columns.py

@@ -19,7 +19,7 @@ def __init__(self, columns=None, random_state=0):
                         random_state=random_state)

    def _check_input_for_columns(self, X):
-        cols = self.parameters["columns"] or []
+        cols = self.parameters.get("columns", [])


Just a style tweak, not important

dsherry · 2020-06-12T00:59:32Z

evalml/pipelines/components/utils.py

    OneHotEncoder,
    PerColumnImputer,
    RFClassifierSelectFromModel,
    RFRegressorSelectFromModel,
    SimpleImputer,
-    StandardScaler,
-    Transformer


Its good to remove these -- they shouldn't show up in all_components() because they're base classes and would never be used in a pipeline directly. Right now, any subclass of ComponentBase which is imported in this file will be included in all_components().

codecov · 2020-06-12T01:01:12Z

Codecov Report

Merging #847 into master will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #847   +/-   ##
=======================================
  Coverage   99.68%   99.68%           
=======================================
  Files         195      195           
  Lines        7699     7738   +39     
=======================================
+ Hits         7675     7714   +39     
  Misses         24       24

Impacted Files	Coverage Δ
...valml/pipelines/components/estimators/estimator.py	`100.00% <ø> (ø)`
...s/components/estimators/regressors/et_regressor.py	`100.00% <ø> (ø)`
...s/components/estimators/regressors/rf_regressor.py	`100.00% <ø> (ø)`
...l/pipelines/components/transformers/transformer.py	`100.00% <ø> (ø)`
evalml/tests/component_tests/test_en_classifier.py	`100.00% <ø> (ø)`
evalml/tests/component_tests/test_en_regressor.py	`100.00% <ø> (ø)`
evalml/tests/component_tests/test_et_classifier.py	`100.00% <ø> (ø)`
evalml/tests/component_tests/test_et_regressor.py	`100.00% <ø> (ø)`
...ification_pipeline_tests/test_en_classification.py	`100.00% <ø> (ø)`
...ification_pipeline_tests/test_et_classification.py	`100.00% <ø> (ø)`
... and 24 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2b54ced...ad7dae3. Read the comment docs.

dsherry · 2020-06-12T01:02:04Z

evalml/pipelines/components/utils.py

@@ -47,7 +43,7 @@ def _components_dict():
    components = dict()
    for _, obj in inspect.getmembers(sys.modules[__name__], inspect.isclass):
        params = inspect.getargspec(obj.__init__)
-        if issubclass(obj, ComponentBase):
+        if issubclass(obj, ComponentBase) and obj is not ComponentBase:


This excludes ComponentBase from all_components()

dsherry · 2020-06-12T01:03:50Z

evalml/tests/component_tests/test_components.py

+        assert defaults.pop(-1) == 0
+
+        expected_parameters = {arg: default for (arg, default) in zip(args, defaults)}
+        assert parameters == expected_parameters


This is the main test. It introspects on the __init__ of all components listed in all_components(), and expects they've saved all args in parameters except for random_state, which should be the last __init__ arg.

I'd love to find a way to get this code into a metaclass so it can be used to raise exceptions at class-definition-time when a user defines a custom component. I was working on that a couple months ago, but... it's hard!! 😂

dsherry · 2020-06-12T01:05:22Z

evalml/tests/component_tests/test_components.py

+        component2 = component_class(**parameters)
+        parameters2 = component2.parameters
+
+        assert parameters == parameters2


This test ensures that instantiating a component with no parameters and instantiating a component with default parameters produces the same parameters attached to the instance.

We need to add default_parameters, will file that.

eccabay · 2020-06-12T12:38:38Z

evalml/pipelines/components/estimators/estimator.py

-    """A component that fits and predicts given data"""
+    """A component that fits and predicts given data
+
+    To implement a new Transformer, define your own class which is a subclass of Transformer, including


Do you mean Estimator here?

Yep, thanks, good eye, will fix!

jeremyliweishih

LGTM. Will be great to get this in for to assist #842 and #474.

jeremyliweishih · 2020-06-12T14:34:18Z

evalml/pipelines/components/estimators/classifiers/catboost_classifier.py

-        cb_classifier = catboost.CatBoostClassifier(**parameters,
+        # catboost will choose an intelligent default for bootstrap_type, so only set if provided
+        cb_parameters = copy.copy(parameters)
+        if bootstrap_type is None:


I like this inversion!

dsherry added 18 commits June 11, 2020 10:22

Ensure names used in all_components dict match component names

4568671

Make component_obj and random_state optional params for ComponentBase…

058af82

… init

Add unit test of params

2c28421

Add default args for components, params getter

1178b72

Exclude base classes from components list

24922da

Changes to component tests after setting default args

35a49a8

lint

a4e23b4

Style tweak in drop columns

e0bf0df

Add unit test of default parameter values for all components and esti…

0a1b779

…mators

Update test impl

7af7cc4

Save all user-provided inputs to components in the parameters dict

9660519

Update failing test

fbbc749

Update component tests

943cdaa

Fix pipeline tests

c7690b2

Add comments to estimator/transformer

09241ef

Don't initialize with dict in signature

e142f81

Changelog

3cfe6ff

Add comments

ad7dae3

dsherry commented Jun 12, 2020

View changes

dsherry marked this pull request as ready for review June 12, 2020 01:12

dsherry requested review from angela97lin, eccabay and jeremyliweishih June 12, 2020 01:12

dsherry mentioned this pull request Jun 12, 2020

Clone Pipelines #842

Merged

auto-assign bot assigned dsherry Jun 12, 2020

dsherry mentioned this pull request Jun 12, 2020

Ability to clone components and pipelines #534

Closed

eccabay reviewed Jun 12, 2020

View changes

jeremyliweishih approved these changes Jun 12, 2020

View changes

dsherry merged commit bcc9242 into master Jun 12, 2020
2 checks passed

dsherry deleted the ds_522_standardize_components branch June 12, 2020 14:34

angela97lin mentioned this pull request Jun 30, 2020

Release v0.11.0 #901

Merged

Component parameters: save all fields for builtin components #847

Component parameters: save all fields for builtin components #847

dsherry commented Jun 12, 2020

dsherry Jun 12, 2020

dsherry Jun 12, 2020 •

edited

codecov bot commented Jun 12, 2020 •

edited

dsherry Jun 12, 2020

dsherry Jun 12, 2020 •

edited

dsherry Jun 12, 2020

dsherry Jun 12, 2020

eccabay Jun 12, 2020

dsherry Jun 12, 2020

jeremyliweishih left a comment

jeremyliweishih Jun 12, 2020

Component parameters: save all fields for builtin components #847

Component parameters: save all fields for builtin components #847

Conversation

dsherry commented Jun 12, 2020

dsherry Jun 12, 2020

Choose a reason for hiding this comment

dsherry Jun 12, 2020 • edited

Choose a reason for hiding this comment

codecov bot commented Jun 12, 2020 • edited

Codecov Report

dsherry Jun 12, 2020

Choose a reason for hiding this comment

dsherry Jun 12, 2020 • edited

Choose a reason for hiding this comment

dsherry Jun 12, 2020

Choose a reason for hiding this comment

dsherry Jun 12, 2020

Choose a reason for hiding this comment

eccabay Jun 12, 2020

Choose a reason for hiding this comment

dsherry Jun 12, 2020

Choose a reason for hiding this comment

jeremyliweishih left a comment

Choose a reason for hiding this comment

jeremyliweishih Jun 12, 2020

Choose a reason for hiding this comment

dsherry Jun 12, 2020 •

edited

codecov bot commented Jun 12, 2020 •

edited

dsherry Jun 12, 2020 •

edited