05.4-agnostic-permfeatimp.Rmd

## Permutation Feature Importance

The permutation feature importance measurement was  introduced for RandomForests by @breiman2001random.
It works by measuring what happens to the model performance when you permute each feature.
A big loss in performance means a big feature importance.
The idea of the permutation of features is per se model-agnostic, only the out-of-bag (=datapoints not used for model fitting) estimation scheme is specific for bootstrapped ensemble methods.
Permutation feature importance can be used for any model when a hold-out dataset, instead of out-of-bag samples is used.
Of course you could also use the training data, but you risk
getting variable importance measures that overfit your data, since the model was already trained on it.

The algorithm [@breiman2001random], generalised for model-agnostic application, is defined as:

Input: Trained model $\hat{f}$, hold-out dataset $D$, number of permutations $n_{perm}$

1. Estimate performance $Perf$ of $\hat{f}$ on dataset $D$ (e.g. mean squared error)
2. For each feature $j \in 1, \ldots, p$ do
    - For $i \in 1,\ldots , n_{perm}$
        - Generate data $D_{j_{perm}}$ by permuting feature $X_j$ in data $D$. This breaks the association between $X_j$ and $Y$.
        - Estimate performance $Perf_{i,j_{perm}}$ of $\hat{f}$ on dataset $D_{j_{perm}}$
        - Calculate permutation feature importance $FI_i(X_j) = Perf_{i,j_{perm}} - Perf$
    - Calculate mean variable importance for the sample: $FI(X_j) = \frac{1}{n_{perm}}\sum_{i=1}^{n_{perm}} FI_i(X_j)$
3. Sort variables by descending $FI$.

The feature with the highest importance measure $FI$ makes the most difference in performance globally in your model.
I haven't found any paper that generalises permutation feature importance, so that it can be applied model-agnostic.
Please drop me a mail if you know about model-agnostic feature importance.