[ML] Improvements to the outlier influence calculation #2256
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tveasey
commented
Sep 5, 2022
| double logCdfComplement{common::CTools::fastLog( | ||
| std::max(cdfComplement, std::numeric_limits<double>::min()))}; | ||
| return P_OUTLIER.value(logCdfComplement); | ||
| }; |
There was a problem hiding this comment.
Note that this represents a small change to how we compute outlier scores for extreme outliers. Before things would saturate which means we couldn't reliably compute the derivative of the overall score w.r.t. changes in feature values. This extends the range to std::numeric_limits<double>::min(). It also avoids making the function as flat around P = 0.5 for the same reason. The resulting scores are still broadly similar.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, we compute influence of features on the outlier score by searching for the minimum value of the outlier score allowing one coordinate to vary at a time. This works well if the data aren't projected, but when the data are high dimensional we randomly project them. We then have to work out how changes in projected coordinates should be attributed to the original features. This is only possible approximately and can lead to confusing results.
The normal definition of influence is the (Gateaux) derivative of the outlier score with respect to the feature values. This has the advantage that we can simply compute the change in the projection space for small changes in a single feature value. This changes switches to use this definition. For the unprojected case the results are broadly similar: the unit tests pass with minor tweaks. However, this fixes confusing results in the other case.