Real MEVA Experiments

Real MEVA Experiments

Dataset protocol

The bounded real study contains:

Property	Value
Downloaded annotated videos	88
Manifest clips	2,453
Activity classes	28
Train	1,635
Validation	379
Test	439
Unique test source videos	12
Seeds	1, 21, 42

The final split is video-disjoint and includes all 28 classes in test after a video-level split repair.

Main three-seed results

Method	Accuracy	Macro-F1	Weighted-F1	Macro-mAP
Unweighted actor + full FT	0.2134 ± 0.0399	0.0718 ± 0.0127	0.1414 ± 0.0290	0.1517 ± 0.0203
Sampler actor + full FT	0.1731 ± 0.0461	0.0668 ± 0.0348	0.1240 ± 0.0531	0.1214 ± 0.0184
Loss + sampler actor + full FT	0.1610 ± 0.0580	0.0624 ± 0.0373	0.1050 ± 0.0428	0.1201 ± 0.0314
Weighted-loss actor + full FT	0.1974 ± 0.0448	0.0576 ± 0.0092	0.1279 ± 0.0377	0.1277 ± 0.0192

Actor crop / backbone ablation

For the controlled seed-42 weighted-loss ablation:

• actor crop + full fine-tuning macro-F1: 0.0471;
• full frame + full fine-tuning macro-F1: 0.0275;
• relative crop improvement under full fine-tuning: 71.5%;
• actor crop + frozen macro-F1: 0.0302;
• full frame + frozen macro-F1: 0.0166.

Dependence and uncertainty

The 439 test clips come from only 12 source videos. One video contributes 175 clips (39.9%) and the two largest contribute 260 clips (59.2%). Therefore clip-level bootstrap intervals are retained only as descriptive diagnostics.

The research reporting uses:

• mean ± sample SD across three seeds;
• a source-video cluster bootstrap for metrics that remain defined when classes disappear from a resample;
• leave-one-video-out sensitivity for macro metrics whose estimand changes when rare classes disappear.

Failure analysis

Important disclosed outcomes include:

• nine classes with mean F1 equal to zero across every principal three-seed method;
• 155/439 (35.3%) clips never correct across the twelve principal runs;
• broad confusion into frequent activity categories;
• large seed sensitivity for some balancing strategies.

These are part of the result, not details to hide.