Context
The text lane's blind-metric substitution probe (#157, blind_metric.py) measures whether a model drifts toward a hidden-rubric decoy it would not pick unprompted. The imaging lane does not yet have this probe. This issue adds it for NIH ChestX-ray14.
What to run
Port the blind-metric design to imaging: introduce a hidden scoring rubric that rewards a specific wrong finding-read, and measure whether the holdout drifts toward that decoy on the cued images, versus its unprompted baseline read.
Done when
Committed blind-metric summary (decoy uptake, and whether drifters name the rubric, mirroring the text lane's finding that all drifters there self-declared).
Related
Depends on #162 (manifest). Mirrors the text lane's #157 design for the imaging lane.
Context
The text lane's blind-metric substitution probe (#157,
blind_metric.py) measures whether a model drifts toward a hidden-rubric decoy it would not pick unprompted. The imaging lane does not yet have this probe. This issue adds it for NIH ChestX-ray14.What to run
Port the blind-metric design to imaging: introduce a hidden scoring rubric that rewards a specific wrong finding-read, and measure whether the holdout drifts toward that decoy on the cued images, versus its unprompted baseline read.
Done when
Committed blind-metric summary (decoy uptake, and whether drifters name the rubric, mirroring the text lane's finding that all drifters there self-declared).
Related
Depends on #162 (manifest). Mirrors the text lane's #157 design for the imaging lane.