Skip to content

ENH add metadata routing to ColumnTransformer - #27005

Merged
glemaitre merged 16 commits into
scikit-learn:mainfrom
adrinjalali:slep6/columntransformer
Aug 21, 2023
Merged

ENH add metadata routing to ColumnTransformer#27005
glemaitre merged 16 commits into
scikit-learn:mainfrom
adrinjalali:slep6/columntransformer

Conversation

@adrinjalali

@adrinjalali adrinjalali commented Aug 3, 2023

Copy link
Copy Markdown
Member

Towards #22893

This adds metadata routing to ColumnTransformer.

It also adds some docstrings to private methods and does a a bit of refactoring / clean up.

Fixes #19465
Fixes #24490

@github-actions

github-actions Bot commented Aug 3, 2023

Copy link
Copy Markdown

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: 01daf0c. Link to the linter CI: here

@adrinjalali
adrinjalali marked this pull request as ready for review August 7, 2023 15:58
@adrinjalali

Copy link
Copy Markdown
Member Author

@OmarManzoor OmarManzoor left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @adrinjalali . Here are a few comments considering files other than the tests.

Comment thread sklearn/compose/_column_transformer.py Outdated
Comment thread sklearn/compose/_column_transformer.py Outdated
Comment thread sklearn/utils/_metadata_requests.py Outdated
Comment thread sklearn/compose/_column_transformer.py
Comment thread sklearn/compose/_column_transformer.py Outdated
@glemaitre
glemaitre self-requested a review August 11, 2023 20:37

@glemaitre glemaitre left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Only some nitpicks.

Comment thread sklearn/compose/_column_transformer.py Outdated
self._name_to_fitted_passthrough = {}

for name, old, column, _ in self._iter():
for name, old, column, _ in self._iter(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this function was complex to understand, do you mind putting a docstring (at least a summary).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have another branch where I'm doing more docstrings and some refactoring, adding the docstring there. Wanted to keep this PR somewhat small.

_transform_one.

fitted : bool
Used to get an iterable of transformers. If True, use the fitted

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could almost infer this parameter from the func callable: if requesting _transform then you expect it to be fitted=True, if _fit in the name then fitted=False. I don't know if we having the keyword makes more explicit in some ways?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair enough, remember to leave this comment in the upcoming refactoring PR please 😁

Comment thread sklearn/compose/_column_transformer.py Outdated
Comment thread sklearn/compose/_column_transformer.py
Comment thread sklearn/compose/tests/test_column_transformer.py Outdated
Comment thread sklearn/compose/tests/test_column_transformer.py Outdated
Comment thread sklearn/pipeline.py

@OmarManzoor OmarManzoor left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small suggestion, otherwise LGTM. Thanks @adrinjalali

Comment thread sklearn/compose/tests/test_column_transformer.py Outdated
Co-authored-by: Omar Salman <omar.salman@arbisoft.com>
Comment thread sklearn/compose/_column_transformer.py Outdated
fitted=False, column_as_strings=False, replace_strings=True
):
if step is None:
continue

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be safer to have a test for this uncovered line.

@glemaitre glemaitre left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart of the missing line, LGTM.

@adrinjalali

Copy link
Copy Markdown
Member Author

Apart of the missing line, LGTM.

@glemaitre doesn't seem like we're checking for a step being None anywhere else, so I removed the line.

@glemaitre

Copy link
Copy Markdown
Member

Fine with me if the tests are passing. Activating the auto-merge here.

@glemaitre
glemaitre enabled auto-merge (squash) August 21, 2023 12:00
@glemaitre
glemaitre merged commit 96b5814 into scikit-learn:main Aug 21, 2023
TamaraAtanasoska pushed a commit to TamaraAtanasoska/scikit-learn that referenced this pull request Aug 21, 2023
Co-authored-by: Omar Salman <omar.salman@arbisoft.com>
@adrinjalali
adrinjalali deleted the slep6/columntransformer branch August 24, 2023 11:16
akaashpatelmns pushed a commit to akaashp2000/scikit-learn that referenced this pull request Aug 25, 2023
Co-authored-by: Omar Salman <omar.salman@arbisoft.com>
avm19 pushed a commit to avm19/scikit-learn that referenced this pull request Sep 13, 2023
REDVM pushed a commit to REDVM/scikit-learn that referenced this pull request Nov 16, 2023
Co-authored-by: Omar Salman <omar.salman@arbisoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add **fit_params to sklearn.compose.ColumnTransformer().fit() Passing fitting parameters to transformers of ColumnTransformer

3 participants