extraction: yield on an identical source varies ~40% across runs, with no ground truth to judge it

[jasonssdev]

Summary

Extraction yield on an identical source varies by ~40% between runs, and nothing
in the output tells the user which run to trust.

Evidence

transcription1.md, byte-identical, three runs:

Run	Objects	Decisions
2026-08-11	12	1
2026-08-13	15	3
2026-08-14	9	1

Deliberately not filed as a regression

Fewer objects is not automatically worse. The 08-13 run included visible
fragmentation — generación-aumentada-por-recuperación-rag alongside
sistema-de-generación-aumentada-por-recuperación-rag,
metodología-de-experimentación alongside
metodología-de-experimentación-con-modelos. If the 08-14 run's nine objects are
nine distinct subjects, it is the better result.

The problem is that the count alone cannot distinguish these cases, for the
maintainer or for a user. Object count was a usable proxy while the failure mode
was catastrophic (a single object restating the filename is obviously wrong). It
stops being usable once extraction works.

What would resolve it

Ground truth. A fixture source with a human-annotated object list, scored for
precision and recall rather than count, and run enough times to report variance
rather than a single number.

Until that exists, "extraction improved" and "extraction regressed" are both
unfalsifiable claims about this pipeline.