Skip to content

fix Sklearnwrapper output for OneHotEncoder and PolynomialFeatures to avoid duplicated features #491

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
solegalli merged 4 commits into from
Aug 30, 2022
+186 −17
Merged

fix Sklearnwrapper output for OneHotEncoder and PolynomialFeatures to avoid duplicated features #491

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4221098
Fix repeated features for OneHotEncoder and PolynomialFeatures
datacubeR Aug 4, 2022
b8a4386
Fixed Tests for SklearnTransformerWrapper using OneHotEncoder and Pol…
datacubeR Aug 7, 2022
c204083
New Tests + Examples for Wrapped OneHotEncoder and Polynomial Features
datacubeR Aug 13, 2022
ec12e8c
Adding last Fixes and Docs
Aug 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions docs/user_guide/wrappers/Wrapper.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,96 @@ to select only a subset of the variables.
X_train_t = selector.transform(X_train.fillna(0))
X_test_t = selector.transform(X_test.fillna(0))

Even though Feature-engine has its own implementation of OneHotEncoder, you may want
to use Scikit-Learn's transformer in order to access different options,
such as drop first Category.
In the following example, we show you how to apply Scikit-learn's OneHotEncoder to a
subset of categories using the :class:SklearnTransformerWrapper().

.. code:: python

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import OneHotEncoder

df = pd.read_csv('https://www.openml.org/data/get_csv/16826755/phpMYEkMl')
X = df
y = df.survived
X_train, X_test, y_train, y_test= train_test_split(X, y, test_size=0.2, random_state=42)

ohe = SklearnTransformerWrapper(OneHotEncoder(sparse=False, drop='first'), variables = ['pclass','sex'])

ohe.fit(X_train)

X_train_transformed = ohe.transform(X_train)
X_test_transformed = ohe.transform(X_test)

@solegalli solegalli Aug 13, 2022

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to add an image or a printout of the result, for example execute X_train_transform.head() and after that show an image of the final df, or instead print(X_train_transform.head()) and then copy the code output in a code block.

@datacubeR datacubeR Aug 13, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

print(X_train_transformed.head())
age fare embarked pclass_2 pclass_3 sex_male
772 17 7.8958 S 0.0 1.0 1.0
543 36 10.5 S 1.0 0.0 1.0
289 18 79.65 S 0.0 0.0 0.0
10 47 227.525 C 0.0 0.0 1.0
147 NaN 42.4 S 0.0 0.0 1.0

print(X_test_transformed.head())
age fare embarked pclass_2 pclass_3 sex_male
1148 35 7.125 S 0.0 1.0 1.0
1049 20 15.7417 C 0.0 1.0 1.0
982 NaN 7.8958 S 0.0 1.0 1.0
808 NaN 8.05 S 0.0 1.0 1.0
1195 NaN 7.75 Q 0.0 1.0 1.0


Let's say you want to use :class:`SklearnTransformerWrapper()` in a more complex
context. As you may note there are `?` signs to denote unknown values. Due to the
complexity of the transformations needed we'll use a Pipeline to impute missing values,
encode categorical features and create interactions for specific variables using
Scikit-Learn's PolynomialFeatures.

.. code:: python

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import Pipeline
from feature_engine.imputation import CategoricalImputer, MeanMedianImputer
from feature_engine.encoding import OrdinalEncoder
from feature_engine.wrappers import SklearnTransformerWrapper

df = pd.read_csv('https://www.openml.org/data/get_csv/16826755/phpMYEkMl')
X = df[['pclass','sex','age','fare','embarked']].replace('?',np.nan)
X[['age', 'fare']] = X[['age', 'fare']].astype('float64')
y = df.survived

X_train, X_test, y_train, y_test= train_test_split(X, y, test_size=0.2, random_state=42)
pipeline = Pipeline(steps = [
('ci', CategoricalImputer(imputation_method='frequent')),
('mmi', MeanMedianImputer(imputation_method='mean')),
('od', OrdinalEncoder(encoding_method='arbitrary')),
('pl', SklearnTransformerWrapper(PolynomialFeatures(interaction_only = True, include_bias=False), variables=['pclass','sex']))
])
pipeline.fit(X_train)
X_train_transformed = pipeline.transform(X_train)
X_test_transformed = pipeline.transform(X_test)

@solegalli solegalli Aug 13, 2022

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great example, thank you!!

Could you add a 5 row display of the resulting dataframe? like in my previous comment.


print(X_train_transformed.head())
age fare embarked pclass sex pclass sex
772 17.000000 7.8958 0 3.0 0.0 0.0
543 36.000000 10.5000 0 2.0 0.0 0.0
289 18.000000 79.6500 0 1.0 1.0 1.0
10 47.000000 227.5250 1 1.0 0.0 0.0
147 29.532738 42.4000 0 1.0 0.0 0.0

print(X_test_transformed.head())
age fare embarked pclass sex pclass sex
1148 35.000000 7.1250 0 3.0 0.0 0.0
1049 20.000000 15.7417 1 3.0 0.0 0.0
982 29.532738 7.8958 0 3.0 0.0 0.0
808 29.532738 8.0500 0 3.0 0.0 0.0
1195 29.532738 7.7500 2 3.0 0.0 0.0

More details
^^^^^^^^^^^^
Expand Down
9 changes: 7 additions & 2 deletions feature_engine/wrappers/wrappers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,7 @@ def transform(self, X: pd.DataFrame) -> pd.DataFrame:
columns=self.transformer_.get_feature_names_out(self.variables_),
index=X.index,
)
X = pd.concat([X, new_features_df], axis=1)
X = pd.concat([X.drop(columns=self.variables_), new_features_df], axis=1)

# Feature selection: transformers that remove features
elif self.transformer_.__class__.__name__ in _SELECTORS:
Expand Down Expand Up @@ -373,7 +373,12 @@ def get_feature_names_out(
added_features = self.transformer_.get_feature_names_out(
self.variables_
)
feature_names = list(self.feature_names_in_) + list(added_features)
original_features = [
feature
for feature in self.feature_names_in_
if feature not in self.variables_
]
feature_names = original_features + list(added_features)
else:
feature_names = list(
self.transformer_.get_feature_names_out(input_features)
Expand Down
104 changes: 89 additions & 15 deletions tests/test_wrappers/test_sklearn_wrapper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@
StandardScaler,
)

from feature_engine.selection import DropFeatures
from feature_engine.wrappers import SklearnTransformerWrapper

_transformers = [
Expand Down Expand Up @@ -176,17 +175,72 @@ def test_wrap_polynomial_features():
)
Xw = tr_wrap.fit_transform(X)

pd.testing.assert_frame_equal(Xw, pd.concat([X, Xt], axis=1))
assert Xw.shape[1] == len(X.columns) + len(tr.get_feature_names_out(varlist))
pd.testing.assert_frame_equal(Xw, pd.concat([X.drop(columns=varlist), Xt], axis=1))
assert Xw.shape[1] == len(X.drop(columns=varlist).columns) + len(
tr.get_feature_names_out(varlist)
)

# when variable list is None
tr_wrap.set_params(variables=None)

Xt = pd.DataFrame(tr.fit_transform(X), columns=tr.get_feature_names_out())
Xw = tr_wrap.fit_transform(X)

pd.testing.assert_frame_equal(Xw, pd.concat([X, Xt], axis=1))
assert Xw.shape[1] == len(X.columns) + len(tr.get_feature_names_out(X.columns))
pd.testing.assert_frame_equal(Xw, Xt)
assert Xw.shape[1] == len(tr.get_feature_names_out(X.columns))


def test_wrap_polynomial_features_get_features_name_out():
X = fetch_california_housing(as_frame=True).frame

varlist = ["MedInc", "HouseAge", "AveRooms", "AveBedrms"]
tr_wrap = SklearnTransformerWrapper(
transformer=PolynomialFeatures(), variables=varlist
)

tr_wrap.fit(X)
expected_features_all = [
"Population",
"AveOccup",
"Latitude",
"Longitude",
"MedHouseVal",
"1",
"MedInc",
"HouseAge",
"AveRooms",
"AveBedrms",
"MedInc^2",
"MedInc HouseAge",
"MedInc AveRooms",
"MedInc AveBedrms",
"HouseAge^2",
"HouseAge AveRooms",
"HouseAge AveBedrms",
"AveRooms^2",
"AveRooms AveBedrms",
"AveBedrms^2",
]
expected_features_varlist = [
"1",
"MedInc",
"HouseAge",
"AveRooms",
"AveBedrms",
"MedInc^2",
"MedInc HouseAge",
"MedInc AveRooms",
"MedInc AveBedrms",
"HouseAge^2",
"HouseAge AveRooms",
"HouseAge AveBedrms",
"AveRooms^2",
"AveRooms AveBedrms",
"AveBedrms^2",
]

assert tr_wrap.get_feature_names_out() == expected_features_all
assert tr_wrap.get_feature_names_out(varlist) == expected_features_varlist


# SimpleImputer
Expand Down Expand Up @@ -271,7 +325,6 @@ def test_sklearn_ohe_object_one_feature(df_vartypes):

ref = pd.DataFrame(
{
"Name": ["tom", "nick", "krish", "jack"],
"Name_jack": [0, 0, 0, 1],
"Name_krish": [0, 0, 1, 0],
"Name_nick": [0, 1, 0, 0],
Expand All @@ -294,8 +347,6 @@ def test_sklearn_ohe_object_many_features(df_vartypes):

ref = pd.DataFrame(
{
"Name": ["tom", "nick", "krish", "jack"],
"City": ["London", "Manchester", "Liverpool", "Bristol"],
"Name_jack": [0, 0, 0, 1],
"Name_krish": [0, 0, 1, 0],
"Name_nick": [0, 1, 0, 0],
Expand All @@ -322,7 +373,6 @@ def test_sklearn_ohe_numeric(df_vartypes):

ref = pd.DataFrame(
{
"Age": [20, 21, 19, 18],
"Age_18": [0, 0, 0, 1],
"Age_19": [0, 0, 1, 0],
"Age_20": [1, 0, 0, 0],
Expand All @@ -342,11 +392,6 @@ def test_sklearn_ohe_all_features(df_vartypes):

ref = pd.DataFrame(
{
"Name": ["tom", "nick", "krish", "jack"],
"City": ["London", "Manchester", "Liverpool", "Bristol"],
"Age": [20, 21, 19, 18],
"Marks": [0.9, 0.8, 0.7, 0.6],
"dob": pd.date_range("2020-02-24", periods=4, freq="T"),
"Name_jack": [0, 0, 0, 1],
"Name_krish": [0, 0, 1, 0],
"Name_nick": [0, 1, 0, 0],
Expand Down Expand Up @@ -402,7 +447,6 @@ def test_sklearn_ohe_with_crossvalidation():
variables=["AveBedrms_cat"],
),
),
("cleanup", DropFeatures(["AveBedrms_cat"])),
("model", Lasso()),
]
)
Expand All @@ -414,6 +458,36 @@ def test_sklearn_ohe_with_crossvalidation():
assert not any([np.isnan(i) for i in results])


def test_wrap_one_hot_encoder_get_features_name_out(df_vartypes):
ohe_wrap = SklearnTransformerWrapper(transformer=OneHotEncoder(sparse=False))
ohe_wrap.fit(df_vartypes)

expected_features_all = [
"Name_jack",
"Name_krish",
"Name_nick",
"Name_tom",
"City_Bristol",
"City_Liverpool",
"City_London",
"City_Manchester",
"Age_18",
"Age_19",
"Age_20",
"Age_21",
"Marks_0.6",
"Marks_0.7",
"Marks_0.8",
"Marks_0.9",
"dob_2020-02-24T00:00:00.000000000",
"dob_2020-02-24T00:01:00.000000000",
"dob_2020-02-24T00:02:00.000000000",
"dob_2020-02-24T00:03:00.000000000",
]

assert ohe_wrap.get_feature_names_out() == expected_features_all


@pytest.mark.parametrize(
"transformer",
[PowerTransformer(), OrdinalEncoder(), MinMaxScaler(), StandardScaler()],
Expand Down