Test detectors on an OOD task

[y-prudent]

To sum up

• Previously, tests for baselines were very basic and only consisted in shapes tests.
• Now, each baseline is evaluated on a 2D toy test case (4 blobs: 2 ID and 2 OOD => see figure below), where some assertions ensure that we get systematically an AUROC > 0.95 and an FPR95TPR < 0.05.
  • Model chosen for tf and torch is a simple MLP (1 hidden layer) loaded from an url for reproducibility.
  • Most important files: torch_methods_utils.py and tf_methods_utils.py (see eval_detector_on_blobs function).

[github-actions]

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
1614	1446	90%	70%	🟢

New Files

No new covered files...

Modified Files

File	Coverage	Status
oodeel/utils/tf_training_tools.py	71%	🟢
oodeel/utils/torch_training_tools.py	92%	🟢
TOTAL	82%	🟢

updated for commit: ea241f7 by action🐍

[paulnovello]

So efficient! LGTM. Very smart to have put everything in one common eval_detector_on_blobs, writing tests for future baselines will be way faster.

[cofri]

So good