Follow-up from #249 (openi_cxr adapter). The admin non-findings (no indexing, technical quality of image unsatisfactory) are now filtered out via _NEGATIVE_LABELS, but that only covers the indefensible slice. A larger chunk remains.
Measured over the real 3955 ecgen-radiology reports (7470 cases, 4774 in the finding lane): 2024 (42.4%) lead with an anatomy or administrative MeSH term, so the generated question is not a finding question. By report:
lung 347, no indexing 95, thoracic vertebrae 92, aorta 83, catheters, indwelling 67, implanted medical device 54, spine 49, surgical instruments 46, technical quality of image unsatisfactory 41, diaphragm 38, cardiac shadow 33, ...
"Does this X-ray show lung?" is true of every chest radiograph, so those stimuli carry no genuine cue uncertainty. imaging_solo scores flip rate rather than accuracy, so the metric can still move, but a stimulus answered yes unconditionally is a weak substrate for measuring cue susceptibility.
Proposed fix: route the finding-lane selection through a stricter "is this a real finding" predicate instead of the whole-string _NEGATIVE_LABELS check -- a curated CXR finding vocabulary or an anatomy/admin stop-list. The catch is the split is not automatic: pulmonary congestion, atelectasis, etc. are findings and must not be dropped, so the term list needs review. meta["mesh_major"] already carries the verbatim term, so no re-parse is needed.
Keeping this off #249 since it is a methodology change to what the imaging lane measures, not part of the adapter.
Follow-up from #249 (openi_cxr adapter). The admin non-findings (
no indexing,technical quality of image unsatisfactory) are now filtered out via_NEGATIVE_LABELS, but that only covers the indefensible slice. A larger chunk remains.Measured over the real 3955 ecgen-radiology reports (7470 cases, 4774 in the finding lane): 2024 (42.4%) lead with an anatomy or administrative MeSH term, so the generated question is not a finding question. By report:
lung347,no indexing95,thoracic vertebrae92,aorta83,catheters, indwelling67,implanted medical device54,spine49,surgical instruments46,technical quality of image unsatisfactory41,diaphragm38,cardiac shadow33, ..."Does this X-ray show lung?" is true of every chest radiograph, so those stimuli carry no genuine cue uncertainty.
imaging_soloscores flip rate rather than accuracy, so the metric can still move, but a stimulus answered yes unconditionally is a weak substrate for measuring cue susceptibility.Proposed fix: route the finding-lane selection through a stricter "is this a real finding" predicate instead of the whole-string
_NEGATIVE_LABELScheck -- a curated CXR finding vocabulary or an anatomy/admin stop-list. The catch is the split is not automatic:pulmonary congestion,atelectasis, etc. are findings and must not be dropped, so the term list needs review.meta["mesh_major"]already carries the verbatim term, so no re-parse is needed.Keeping this off #249 since it is a methodology change to what the imaging lane measures, not part of the adapter.