python: fail fast when the --oom-foreign shim is shadowed (no-op injection)

[devdanzin]

Problem

--oom-foreign preloads a malloc-failure shim (fusil_malloc_shim.c) to inject allocation failures at the C malloc() layer, reaching foreign C libraries (HDF5, zstd, ...). Against a statically-linked AddressSanitizer target this silently does nothing: ASan defines/exports its own malloc in the executable, which sits ahead of LD_PRELOAD in the global symbol scope and shadows the shim. The emitted harness only checks that the shim's control symbol resolves (it does), not that interposition actually happens — so the whole run injects zero failures with no error.

This bit us for real: an h5py ASan fleet ran thousands of sessions finding nothing, because no OOM was ever injected. Proven empirically — on the ASan target the shim's malloc counter stays 0 (never called); on a non-ASan python the identical shim works (4/8 injected failures).

Fix

Add a startup self-check. probe_shim_effective(python, shim) runs a tiny probe in the target interpreter (arm the shim, malloc a few times, read the shim's own allocation counter, which only increments from inside the shim's malloc):

• nonzero count → shim is on the allocation path → True
• zero → shadowed → False
• couldn't run / no marker → None (inconclusive, non-fatal)

The --oom-foreign setup aborts with a loud, actionable ShimShadowedError on a definitive False, instead of running a no-op campaign:

ERROR: [ShimShadowedError] --oom-foreign: the malloc shim is preloaded but does NOT
intercept allocations in the target interpreter (...) -- allocations bypass it, so NO
failures would be injected. The usual cause is a statically-linked AddressSanitizer
target whose own malloc shadows the LD_PRELOAD shim. Use a non-ASan target, or rebuild
with -shared-libasan and set ASAN_OPTIONS=verify_asan_link_order=0. Aborting rather than
running a no-op OOM campaign.

An inconclusive probe is non-fatal (only a definitive count==0 blocks), so probe flakiness can't cause a false abort.

Verification

• End-to-end: aborts on the ASan target; prints Foreign-OOM: shim interception verified and proceeds on a non-ASan target.
• +5 tests (pure _parse_probe_count + skip-guarded probe_shim_effective): test_foreign_oom green (8 tests). Full OOM group green (24). ruff check (whole tree) + ruff format --check clean.

🤖 Generated with Claude Code