Applying log transform to skewed outcome variable

[bhu-strata]

My numerical outcome variable is highly skewed so applying a log transform seems to bring it closer to a normal distribution and improves the EBM training accuracy. It is also seems preferable so that the model training isn't biased by the outliers. However, I am having a hard time reconciling the scores and intercept from the model trained on the log-transformed outcome as they differ greatly (even after exponentiating to un-transform them) from a model trained on the original (i.e. not log-transformed) outcome.

1. Any intuition on whether EBMs should benefit from transforming high skewed data?
2. Any suggestions on how to un-transform the intercept and scores so they can be interpretable in the original outcome space?

[paulbkoch]

Hi @bhu-strata -- EBMs should benefit from this if it fits your outcome. This is known as a link function in other GAM packages. It's something that's in our backlog for implementation, but isn't part of our package yet. See #137.

Unfortunately, to maintain additivity you can't really un-transform the scores in the model itself. This is something shared with other GAM packages. I think practitioners eventually get a feel for how to interpret the scores from various link functions. I only have personal experience with this for logits though, so I'm sort of guessing in that regard.

[paulbkoch]

Closing this issue since we already have an issue tracking alternative link functions.

[paulbkoch]

Hi @bhu-strata -- We recently shipped v0.4.0, which includes alternative objectives with a log link. More details are available in our documentation: https://interpret.ml/docs/ebm.html#explainableboostingregressor