embed is a package that contains extra steps for the
recipes package for embedding predictors into one or more numeric columns. All of the preprocessing methods are supervised.
The steps for categorical predictors are:
step_lencode_mixedestimate the effect of each of the factor levels on the outcome and these estimates are used as the new encoding. The estimates are estimated by a generalized linear model. This step can be executed without pooling (via
glm) or with partial pooling (
lmer). Currently implemented for numeric and two-class outcomes.
keras::layer_embeddingto translate the original C factor levels into a set of D new variables (< C). The model fitting routine optimizes which factor levels are mapped to each of the new variables as well as the corresponding regression coefficients (i.e., neural network weights) that will be used as the new encodings.
step_woecreates new variables based on weight of evidence encodings.
For numeric predictors:
step_umapuses a nonlinear transformation similar to t-SNE but can be used to project the transformation on new data. Both supervised and unsupervised methods can be used.
Some references for these methods are:
- Francois C and Allaire JJ (2018) Deep Learning with R, Manning
- Guo, C and Berkhahn F (2016) "Entity Embeddings of Categorical Variables"
- Micci-Barreca D (2001) "A preprocessing scheme for high-cardinality categorical attributes in classification and prediction problems," ACM SIGKDD Explorations Newsletter, 3(1), 27-32.
- Zumel N and Mount J (2017) "
data.frameProcessor for Predictive Modeling"
- McInnes L and Healy J (2018) UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- Good, I. J. (1985), "Weight of evidence: A brief survey", Bayesian Statistics, 2, pp.249-270.
To install the package:
install.packages("embed") ## for development version: require("devtools") install_github("tidymodels/embed")