feat: Target Encoder implementation

[jitingxu1]

Definition

Target encoding involves replacing categorical feature values with a numeric representation derived from the target variable. This method aims to capture the relationship between categorical features and the target variable by encoding categories with their respective impact on the target.

use case

• High Cardinality Features

requirements

• Regression, binary and multiclass classification.
• Handle overfitting
• Handle unknown category, the new category not present in the training dataset.
• Handle missing value

Implementation

Fit

• Treat missing Value
• Use the mean value of target variable for that category for regression task
• Use the conditional probability given that category
• Handle Over-fittings
  • [recommend] Smoothing
  • KFold Target Encoder
  • Leave-one-out
  • Adding Gaussian Noise

Transform

• Unknown category

Issues

• over-fitting
• target leakage

Reference

• Scikit-learn TargetEncoder
• Target encoding done the right way

[jitingxu1]

done