Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make RecordInsightsLOCO perform reasonable calculation on numeric features and fix the name to reflect actual calculation. #557

Open
leahmcguire opened this issue Jun 28, 2021 · 1 comment
Assignees

Comments

@leahmcguire
Copy link
Collaborator

leahmcguire commented Jun 28, 2021

Problem
RecordInsightsLOCO (https://github.com/salesforce/TransmogrifAI/blob/master/core/src/main/scala/com/salesforce/op/stages/impl/insights/RecordInsightsLOCO.scala) is not actually performing LOCO (http://www.stat.cmu.edu/~ryantibs/talks/loco-2018.pdf, https://arxiv.org/pdf/1611.05401.pdf) and should be renamed.

In addition the computation of perturbation measures based on zeroing out values does not make sense for numeric values and should be replaced by computation of the average of the changes measured when the value is perturbed to min, mean, and max.

Solution
Rename transformer.

Grab mean, min and max values for each feature from metadata and use in perturbation calculation instead of zero on appropriate feature types.

Alternatives
Remove the transformer.

Additional context
This transformer was written to see how the model could be applied efficiently across many scores. It was not intended to be a final measure for record insights as written.

@michaelweilsalesforce michaelweilsalesforce self-assigned this Jul 19, 2021
@michaelweilsalesforce
Copy link
Contributor

Need to brainstorm for a better name.

Covariate Perturbation, Covariate Imputation, Incremental Effect on Feature,...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants