-
Notifications
You must be signed in to change notification settings - Fork 630
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable AnomalyScoreMeanVarThreshold based on three-sigma #1052
base: main
Are you sure you want to change the base?
Enable AnomalyScoreMeanVarThreshold based on three-sigma #1052
Conversation
Signed-off-by: Kang Wenjing <wenjing.kang@intel.com>
Signed-off-by: Kang Wenjing <wenjing.kang@intel.com>
Signed-off-by: Kang Wenjing <wenjing.kang@intel.com>
Signed-off-by: Kang Wenjing <wenjing.kang@intel.com>
Signed-off-by: Kang Wenjing <wenjing.kang@intel.com>
test on:
result:
if using adaptive and val_split_mode: same_as_test:
Actually, when using this meanvar threshold, increasing std_dev_ratio to 10, can get a better result for image_AUROC/image_F1Score: There might be two different sweet spots for image/pixel threshold. |
Signed-off-by: Kang Wenjing <wenjing.kang@intel.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. It looks good overall. Just minor comments. Before this can be merged we need to discuss the design for adding more thresholding methods internally.
mean_pred = torch.mean(torch.cat(self.preds, dim=0)) | ||
std_pred = torch.std(torch.cat(self.preds, dim=0)) | ||
if self.meanvar_mode == MeanVarianceMode.MINUS: | ||
self.value = mean_pred - self.std_dev_ratio * std_pred |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As you mentioned in your description this does not make sense in the anomaly context. Maybe we shouldn't support minus
operation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We may add it as a placeholder. If in the future, if we use negative outputs of the model, then minus mode can be used?
Signed-off-by: Kang Wenjing <wenjing.kang@intel.com>
Description
mean - 3 * standard_deviation
ormean + 3 * standard_deviation
, +/- is determined by config. Currently, only the + scenario should be used, because currently this repo determines the result via checking whether the predictions is larger than the threshold.Changes
Checklist