Testing and Quality

Testing and Quality

IncidentGraph uses both software tests and experiment-level validation. These serve different purposes and should not be conflated.

Automated tests

The public repository includes tests for:

• API behavior;
• benchmark behavior;
• MEVA KPF handling;
• synthetic pipeline behavior.

Run:

python -m pytest -q

Experiment quality checks

For activity recognition, report:

• all requested seeds rather than only the best seed;
• macro-F1 and macro-mAP under severe class imbalance;
• weighted-F1 and accuracy for context;
• calibration metrics such as ECE and Brier;
• class-wise failure modes;
• split independence and source-video concentration.

Statistical dependence

The real test set contains 439 clips but only 12 source videos. Do not treat 439 clips as 439 independent incidents in inferential reporting.

Regression checklist

Before merging a meaningful fusion change:

1. run unit/integration tests;
2. re-run the controlled benchmark;
3. compare entity/event/conflict/missing-evidence metrics;
4. confirm provenance coverage remains intact;
5. inspect contradiction alternatives;
6. verify missing-camera cases remain unknown rather than negative;
7. record the exact commit/config used.

Research integrity

Development smoke runs, split-fix experiments and final comparative runs should remain labeled separately. Do not promote debug runs into headline results.