Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion docs/user_guide/selection/DropHighPSIFeatures.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,16 @@ Important

When working with the PSI it is worth highlighting the following:

- The PSI is not symmetric; switching the order of the basis and test dataframes in the PSI calculation will lead to different values.
- The number of bins used to define the distributions has an impact on the PSI values.
- The PSI is a suitable metric for numerical features (i.e., either continuous or with high cardinality).
- For categorical or discrete features, the change in distributions is better assessed with Chi-squared.

he PSI is symmetric. That means that switching the order of the basis and test dataframes
in the PSI calculation will lead to identical values. However, in this implementation, the interval
limits used to calculate the PSI are inferred from the basis dataframe. Hence, switching the
order of the dataframes will lead to different interval limits, which in turn may result in
different PSI values.

Threshold
~~~~~~~~~

Expand Down
5 changes: 3 additions & 2 deletions feature_engine/selection/drop_psi_features.py
Original file line number Diff line number Diff line change
Expand Up @@ -171,8 +171,9 @@ class DropHighPSIFeatures(BaseSelector):

switch: boolean, default=False.
If True, the order of the 2 dataframes used to determine the PSI (basis and
test) will be switched. This is important because the PSI is not symmetric,
i.e., PSI(a, b) != PSI(b, a)).
test) will be switched. This is important because the interval limits used to
calculate the PSI are inferred from the basis dataframe. Hence, changing the
order of the dataframes may lead to different PSI values.

threshold: float, str, default = 0.25.
The threshold to drop a feature. If the PSI for a feature is >= threshold, the
Expand Down