36 new holdout datasets across image and video modalities, expanding coverage of synthetic generation methods and real-world content.
| Released dataset name | Obfuscated holdout name | Modality | Media type |
|---|---|---|---|
sa-1b-shards |
real-image-holdout-975f791a |
image |
real |
sa-1b-shards |
real-image-holdout-9d17e346 |
image |
real |
sa-1b-shards |
real-image-holdout-c8922129 |
image |
real |
sa-1b-shards |
real-image-holdout-82555af6 |
image |
real |
sa-1b-shards |
real-image-holdout-35352d75 |
image |
real |
communityforensics-real |
real-image-holdout-81694902 |
image |
real |
glint360k |
real-image-holdout-30d9329b |
image |
real |
communityforensics-fake |
synthetic-image-holdout-25aca886 |
image |
synthetic |
communityforensics-fake |
synthetic-image-holdout-a5c5ef0f |
image |
synthetic |
imagepulsev2-addremove |
synthetic-image-holdout-dec37878 |
image |
synthetic |
imagepulsev2-change |
synthetic-image-holdout-888684fc |
image |
synthetic |
imagepulsev2-angle |
synthetic-image-holdout-a97c0d16 |
image |
synthetic |
imagepulsev2-crop |
synthetic-image-holdout-284cb6bc |
image |
synthetic |
imagepulsev2-inpaint |
synthetic-image-holdout-c9fbb212 |
image |
synthetic |
imagepulsev2-humanface |
synthetic-image-holdout-016cd50e |
image |
synthetic |
imagepulsev2-style |
synthetic-image-holdout-3139ff06 |
image |
synthetic |
justweirdimages |
synthetic-image-holdout-027002fc |
image |
synthetic |
flux-sd15-ipadapter |
synthetic-image-holdout-a7ea3f58 |
image |
synthetic |
ip-adapter-faceid |
synthetic-image-holdout-3fe582cc |
image |
synthetic |
ip-adapter-faceid |
synthetic-image-holdout-d7ed0468 |
image |
synthetic |
sd35-ip-adapter-faceid |
synthetic-image-holdout-45d8c442 |
image |
synthetic |
sd35-ip-adapter-faceid |
synthetic-image-holdout-7f11881b |
image |
synthetic |
llava-video-178k |
real-video-holdout-43262857 |
video |
real |
omnivideo-100k |
real-video-holdout-ce760bb4 |
video |
real |
omnivideo-test |
real-video-holdout-47243aa8 |
video |
real |
ophora-160k |
real-video-holdout-a799bc65 |
video |
real |
qa-v1 |
real-video-holdout-3c0c27c8 |
video |
real |
qa-test-h264 |
real-video-holdout-de456938 |
video |
real |
deeptumorvqa |
real-video-holdout-2f424a18 |
video |
real |
tiktok-10m |
real-video-holdout-c00a826f |
video |
real |
heico-focus-vqa |
real-video-holdout-f847a136 |
video |
real |
tonystark03-deepfake-real |
real-video-holdout-0cce6c4d |
video |
real |
gen-videos-fastwan2 |
synthetic-video-holdout-c3a396e1 |
video |
synthetic |
gen-videos-hunyuanvideo |
synthetic-video-holdout-d793de35 |
video |
synthetic |
gen-videos-kling |
synthetic-video-holdout-46ee35f7 |
video |
synthetic |
gen-videos-veo3 |
synthetic-video-holdout-dca04654 |
video |
synthetic |
gen-videos-wan2 |
synthetic-video-holdout-eb2d2439 |
video |
synthetic |
gen-videos-sora2 |
synthetic-video-holdout-9a13e6c6 |
video |
synthetic |
cameraclone-dataset |
synthetic-video-holdout-c097c59a |
video |
synthetic |
scene-decoupled-video-dataset |
synthetic-video-holdout-39db0365 |
video |
synthetic |
synthetic-vstat-block-counting |
synthetic-video-holdout-f22b057c |
video |
synthetic |
synthetic-vstat-shuffle-puzzle |
synthetic-video-holdout-c5935b8b |
video |
synthetic |
tonystark03-deepfake-fake |
synthetic-video-holdout-b0fe3d10 |
video |
synthetic |
Adds augmentation robustness tracking to benchmark runs.
- Blends
aug_sn34_scoreintosn34_scoreusingaug_weightinfinalize_run - Carries aug MCC, Brier, and CE forward for persistence alongside the final aug score
- Fixes
.npydouble-extension bug when writing to tmp files - Fixes aug cache to log write errors and unpack video tuples in caller
- Fixes
build_sample_idto includemember_pathfor correct sample identification - Includes aug fields (
aug_sn34_score,aug_binary_mcc,aug_binary_brier,aug_binary_ce) insave_results_to_jsonoutput