Context
The submission commits to Precision@50 but does not pre-agree the format or definition of other evaluation metrics (Recall, AUROC, computational cost units, granularity, explainability) with Cap Vista.
Required
Add a "Validation & Reporting" subsection to Annex A Week 7 deliverables with a pre-agreed metrics table:
| Metric |
Target |
Unit |
Definition |
| Precision@50 |
≥ 0.60 |
Fraction |
Top-50 ranked candidates that are OFAC-listed |
| Recall@200 |
≥ 0.40 |
Fraction |
Fraction of all OFAC-listed vessels in dataset captured in top-200 |
| AUROC |
≥ 0.80 |
0–1 |
Area under ROC curve across all scored vessels |
| Computational cost |
≤ SGD 250/month |
SGD |
Cloud + storage for continuous 15-min re-scoring at PoC scale |
| Granularity |
Per-MMSI + per-feature |
— |
Every prediction tied to a MMSI; top-3 SHAP signals per vessel |
| Explainability |
SHAP + ATT + path |
Qualitative |
Top-3 SHAP attributions, DiD ATT ± 95% CI, ownership graph path per watchlist entry |
Acceptance criteria
Context
The submission commits to Precision@50 but does not pre-agree the format or definition of other evaluation metrics (Recall, AUROC, computational cost units, granularity, explainability) with Cap Vista.
Required
Add a "Validation & Reporting" subsection to Annex A Week 7 deliverables with a pre-agreed metrics table:
Acceptance criteria