[Task]: Implement SimpleNet Paper #1607
-
What is the motivation for this task?Implementation of SimpleNet , one of the sota anomaly detection models published in CVPR 2023. Describe the solution you'd likeAdditional contextNo response |
Beta Was this translation helpful? Give feedback.
Replies: 4 comments
-
|
I agree that we should have this in anomalib. Let's convert this to a feature request |
Beta Was this translation helpful? Give feedback.
-
|
The paper was promising, but it looks like the authors repeatedly evaluated their model on the test data during the training and then reported the best results; see this issue. Results without this scenario-specific tuning can be found in the v2 EfficientAD paper, and they are worse than PatchCore (which was supposedly the main inspiration for SimpleNet), especially on the VisA dataset (87.9 vs 94.3 mean image level AUROC). |
Beta Was this translation helpful? Give feedback.
-
|
That's interesting. @jpcbertoldo proposed an approach to potentially avoid such issues, and create a more consistent benchmarks. For details you could refer to the anomalib PR #1557 and the paper. |
Beta Was this translation helpful? Give feedback.
-
|
Due to the concerns raised for this paper, let's close this |
Beta Was this translation helpful? Give feedback.
The paper was promising, but it looks like the authors repeatedly evaluated their model on the test data during the training and then reported the best results; see this issue. Results without this scenario-specific tuning can be found in the v2 EfficientAD paper, and they are worse than PatchCore (which was supposedly the main inspiration for SimpleNet), especially on the VisA dataset (87.9 vs 94.3 mean image level AUROC).