Compositions return individual steps as part of get_params

[mfeurer]

Describe the bug

When calling get_params on an Pipeline or a ColumnTransformer, it returns the steps as parameters as well. However, the docs for get_params says that only the arguments to __init__ will be returned. By having additional entries here, the output of get_params can no longer be used to call __init__.

Steps/Code to Reproduce

import sklearn.preprocessing
from sklearn.compose import ColumnTransformer

model_original = sklearn.pipeline.Pipeline(
    steps=[("transform", 'passthrough')]
)
print(model_original.get_params().keys())
ct = ColumnTransformer(
    [("continous", "passthrough", [])]
)
print(ct.get_params().keys())

params = ct.get_params()
ColumnTransformer(**params)

Expected Results

dict_keys(['memory', 'steps', 'verbose'])
dict_keys(['n_jobs', 'remainder', 'sparse_threshold', 'transformer_weights', 'transformers', 'verbose'])

Actual Results

dict_keys(['memory', 'steps', 'verbose', 'transform'])
dict_keys(['n_jobs', 'remainder', 'sparse_threshold', 'transformer_weights', 'transformers', 'verbose', 'continous'])
Traceback (most recent call last):
  File "/home/feurerm/sync_dir/projects/openml/python/local/bug3.py", line 14, in <module>
    ColumnTransformer(**params)
  File "/home/feurerm/miniconda/3-4.5.4/envs/openml/lib/python3.7/site-packages/sklearn/utils/validation.py", line 72, in inner_f
    return f(**kwargs)
TypeError: __init__() got an unexpected keyword argument 'continous'

Versions

Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0]
NumPy 1.17.0
SciPy 1.3.0
Scikit-Learn 0.23.2

[mfeurer]

It appears that deep=False turns off this behavior:

import sklearn.preprocessing
from sklearn.compose import ColumnTransformer

model_original = sklearn.pipeline.Pipeline(
    steps=[("transform", 'passthrough')]
)
print(model_original.get_params(deep=False).keys())
ct = ColumnTransformer(
    [("continous", "passthrough", [])]
)
print(ct.get_params(deep=False).keys())

params = ct.get_params(deep=False)
ColumnTransformer(**params)

However, this appears to be inconsistent with the remaining API, for example grid search or other meta estimators such as AdaBoost, where the sub-component is passed as an argument.

[glemaitre]

I think that what we attend with get_params and set_params is the following:

ct = ColumnTransformer(
    [("continous", "passthrough", [])]
)
print(ct.get_params(deep=False).keys())

params = ct.get_params(deep=False)
ct.set_params(**params)

Therefore, if you want a new estimator

ct = ColumnTransformer(
    [("continous", "passthrough", [])]
)
print(ct.get_params(deep=False).keys())

params = ct.get_params(deep=False)
ct_clone = clone(ct)
ct_clone.set_params(**params)

[glemaitre]

What I mean by that is that considering the following snippet:
`

In [4]: model_original.get_params()                                                  
Out[4]: 
{'memory': None,
 'steps': [('clf', LogisticRegression())],
 'verbose': False,
 'clf': LogisticRegression(),
 'clf__C': 1.0,
 'clf__class_weight': None,
 'clf__dual': False,
 'clf__fit_intercept': True,
 'clf__intercept_scaling': 1,
 'clf__l1_ratio': None,
 'clf__max_iter': 100,
 'clf__multi_class': 'auto',
 'clf__n_jobs': None,
 'clf__penalty': 'l2',
 'clf__random_state': None,
 'clf__solver': 'lbfgs',
 'clf__tol': 0.0001,
 'clf__verbose': 0,
 'clf__warm_start': False}

The job of get_params is to show how to set the underlying parameters of the LogisticRegression. But these parameters cannot be passed directly to __init__ of the Pipeline. However, set_params will do the job nicely.

Do you think that updating the documentation mentioning the get_params -> clone -> set_params to achieve your use-case and making the documentation less ambiguous would be enough @mfeurer? Basically, I am not sure about the feasability of get_params -> __init__.

[glemaitre]

It appears that deep=False turns off this behavior. I have to check but it could be what clone is doing internally`.

[mfeurer]

The job of get_params is to show how to set the underlying parameters of the LogisticRegression. But these parameters cannot be passed directly to init of the Pipeline.

I agree on that. But again, the issue is that get_params actually return 'clf' as an entry in the dictionary, which is neither a parameter nor a parameter to a submodel. From the documentation the classifier should only be accessible via the steps list.

The questions is whether this is the desired behavior. If yes, I think updating the documentation would be good to explain this behavior.

My actual use case was parsing parameters (and not creating models), so I just created the above example to demonstrate the issue. Going with deep=False actually allows me to proceed.

Actually, this issue has another interesting implication. This here works:

from sklearn.compose import ColumnTransformer

ct = ColumnTransformer(
    [("continous", "passthrough", [])]
)

params = ct.get_params()

print(params)
# params['continuous'] = 'drop'
ct.set_params(**params)

while this does not:

from sklearn.compose import ColumnTransformer

ct = ColumnTransformer(
    [("continous", "passthrough", [])]
)

params = ct.get_params()

print(params)
params['continuous'] = 'drop'
ct.set_params(**params)

although I guess it would be sufficient to update the error message here, which says:

ValueError: Invalid parameter continuous for estimator ColumnTransformer(transformers=[('continous', 'passthrough', [])]). Check the list of available parameters with `estimator.get_params().keys()`.

[glemaitre]

But the error message is fine. You try to set "continuous" while defining "continous". I think this is just a spelling mistake there, isn't it?

[glemaitre]

I agree on that. But again, the issue is that get_params actually return 'clf' as an entry in the dictionary, which is neither a parameter nor a parameter to a submodel. From the documentation the classifier should only be accessible via the steps list.

OK I see. This would probably make sense to remove it. We would need a deprecation warning thought. @jnothman Do you recall why the submodel would be exposed with get_params?

[glemaitre]

Uhm looking at the documentation again, the docstring of get_params seems to be up-to-date.

If True, will return the parameters for this estimator and contained subobjects that are estimators.

So we should probably update only the developer guide. But I am still intrigued to know the use-case.

[mfeurer]

But the error message is fine. You try to set "continuous" while defining "continous". I think this is just a spelling mistake there, isn't it?

That's embarrassing. There's actually no issue here.

OK I see. This would probably make sense to remove it.

I'm not sure if it's necessary, I think it would be sufficient to update the docs (see my next answer).

Uhm looking at the documentation again, the docstring of get_params seems to be up-to-date.

Hm, probably I interpreted this wrong: I was reading that it only return parameters. But yes, one can also interpret the sentence as you did. The question now is what the author of this sentence intended and whether it can be made unambiguous.

But I am still intrigued to know the use-case.

My use case was parsing the parameters of a model for the Python API to OpenML, and I was surprised to see parts of the pipeline returned. The example I gave had nothing to do with what I'm trying to achieve here, but merely to show what's happening.

[glemaitre]

But I am still intrigued to know the use-case.

My use case was parsing the parameters of a model for the Python API to OpenML, and I was surprised to see parts of the pipeline returned. The example I gave had nothing to do with what I'm trying to achieve here, but merely to show what's happening.

Oh, I meant the use-case on the scikit-learn side and why we exposed the sub-estimator :) That's why I pinged @jnothman :)