docs(adr): propose observation identity for parsed nodes (ADR 0019)

[willhea]

Related issue

No closing keyword. This proposes a decision rather than completing a tracked task, and
there is no issue whose definition of done it satisfies.

Refs #175 (the epic whose matching work depends on it), refs #203 (Study 2, whose labeled
dataset is the artifact this record is about), refs #486 (which asks for the mechanism
this record licenses).

What does this change?

Several artifacts in this repository record a human or measured judgment about a parsed
node of a bill: the hand-labeled matching answer key, the PDF anchor goldens, the
extraction goldens. Each one has to name the node it is talking about, and today the
answer key names it by its text.

That does not work, and this pull request adds the measurement showing why.
Appropriations bills are assembled from repeated boilerplate, so the same body text
appears at several different places in one document. On the committed corpus (58 XML
documents, 49,747 nodes):

documents containing at least one duplicated body text	23
distinct body texts occurring more than once	385
node occurrences inside a duplicate group	1,041
largest multiplicity, one text in one document	9

The direction of that failure is what makes it serious. Matching a stored text to "the
node it came from" succeeds against the wrong twin, so a lookup that should miss instead
appears to hit. Every collapse makes the thing being measured look better than it is.

This has already happened. tests/data/similarity_labels.json stores its texts verbatim
and carries no record of which document, which parser or which node it meant. Three of
its twelve pairs no longer correspond to any node the parser emits, and the script that
would rebuild it cannot run at all. Its own test stays green throughout, because it
scores the stored string against the stored score, which is true by construction.

Nothing here means the legislation changed or that the human judgments were wrong. The
source XML is byte-identical to what entered the repository. What moved is how the parser
divides that XML into nodes, which re-segments the unit the judgment was about. The
fixture had no way to say which parse it meant.

The decision is to identify a parsed observation by (source digest, parser revision, node address), and to keep three questions permanently apart:

question	how it is answered	may two distinct nodes share it?
observation identity — which parsed node is this?	source digest + parser revision + node address	No. Asserted, not assumed.
content integrity — is this still the same text?	a hash of the text	Yes, routinely. That is the finding above, not a defect.
cross-version identity — is this the same provision as that one?	the human's or the matcher's ruling	It is an output, never an input key.

Three details worth a reviewer's attention:

• The address is the node's ordinal in the emitted sequence. Because the key is
  already scoped by source digest and parser revision, a change to either produces a
  different observation identity by construction, so the address only has to designate one
  node within one source under one parser revision. An earlier draft used the XML element
  id; review asked for that to be falsified rather than defended, and it did not survive.
  The element id's uniqueness is an empirical property of the published markup that we can
  only sample, while a list index is unique by construction. The element id is still
  recorded beside the ordinal, for tracing an observation back to the source document, but
  correctness no longer rests on it. The full comparison is in the discussion below.
• The parser revision is derived from the code, not declared. A declared field records
  an intention rather than a fact, and a field that names its own correctness while being
  checked by nobody is decorative. A git commit is also rejected: it moves when
  documentation changes, and it does not move for an uncommitted edit to the parser,
  which is the direction that matters. The record states this as a requirement and leaves
  the hashing mechanism as an implementation detail.
• This is identity within one parse, not across versions. Neither an ordinal nor an
  element id is a cross-version match key, and the record says so explicitly. Treating a
  bill's own numbering as a stable identity is the mistake the matching research exists to
  avoid.

Files

• docs/decisions/0019-observation-identity.md — the proposed record. Context, decision,
  rejected alternatives, consequences, what stays undecided, its relationship to ADR 0009,
  and the invariants and tests it implies.
• scripts/probe_observation_identity.py — the evidence. Read-only, defaults to the
  committed corpus so it needs no downloads, and reports all three candidate address
  fields rather than only the one the record adopts.
• Index entries in AGENTS.md, docs/decisions/README.md and scripts/README.md.

How to test

The record's numbers are reproducible rather than transcribed:

uv run python scripts/probe_observation_identity.py tests/corpus

That prints the table above plus the three supporting results the record relies on:

• the bill's own section addressing is duplicated in 32 of the 58 documents, so it is a
  grouping key rather than an identity;
• parsing is deterministic, which is what the ordinal address requires. Re-run it with a
  different PYTHONHASHSEED and the sequence digest must not move. It does not, across
  three seeds and two corpus roots;
• the element id is unique and non-empty everywhere measured, but only 49,603 of 49,747
  ids are actually present in the source bytes. The other 144 are synthesized by the
  parser, which is why "traceable back to the document" is recorded as a partial property
  rather than an absolute one.

Three claims were confirmed by running them rather than by reading:

• scripts/build_similarity_labels.py, which rebuilds the answer key, exits on
  ImportError: cannot import name '_MOVE_THRESHOLD' from 'deltatrack.diff_bill'. The
  names moved in an earlier refactor and the script did not move with them.
• The determinism check is proven able to fail: swapping two adjacent nodes moves the
  digest, while a digest over the node set stays green. A determinism result that could
  never go red would be indistinguishable from a check that reads nothing.
• The element id's uniqueness could not be broken on either corpus root. The case for the
  ordinal is not that the element id fails; it is that its uniqueness is contingent where
  a list index's is not.

Checklist

• Linked the issue above (Closes #...) — n/a, deliberately. This proposes a
  decision; no tracked issue's definition of done is met by merging it. Refs lines above.
• Ran the CI gates locally and they pass — gates 1, 2, 3 and 5. Gate 4 (browser) was
  not run; it needs a Playwright browser download this environment does not have, and the
  change touches no web surface. Results below.
• New or changed behavior has tests — n/a. No behavior changes: no engine module is
  touched, the new script is standalone and imported by nothing, and the canonical diff
  contract is untouched. The record's invariants describe tests to be written when it is
  implemented, and it names one that will start red.
• For a bug fix: the test fails without the fix — n/a, not a bug fix.
• Disclosed AI assistance below

Gate results

gate	result
1. ruff check .	pass
2. ruff format --check .	pass, 197 files
3. pytest -m "not slow and not browser"	1666 passed, 5 skipped, 15 xfailed. The skips are the usual absent-local-corpus ones.
4. pytest -m browser	not run (no Playwright browser here; no web surface touched)
5. pytest -m slow --deselect tests/test_govinfo_corpus_parity.py	exit 0. 1249 passed, 27 skipped. The skips are content skips on shell and procedural bill versions that carry no dollar amounts, and no skip ceiling was exceeded.

Known limitations

The record is deliberately narrow, and names four things it does not settle:

• Whether this provenance belongs in the published diff contract. It does not today,
  and adding it needs a consumer first.
• Whether the PDF pipeline's emission order is deterministic. The ordinal generalizes
  to PDF where an element id could not, and that is one reason it was chosen. But
  determinism has been measured on the XML side only, and the record does not assume the
  PDF result. The same test is required before any artifact addresses a PDF block this way.
• Whether and when the committed answer key is rewritten to carry the new fields.
  Rewriting a fixture that encodes human rulings is a maintainer decision, and three of its
  records need legislative adjudication regardless.
• What happens when drift is detected on a committed fixture: quarantine for
  re-review, or hard failure.

One dependency is worth stating plainly: implementing this record needs the answer-key
rebuild script repaired, and that repair currently exists only on the provision-matching
research branch.

AI assistance

Drafted with Claude Code. The measurements were produced by running the committed probe,
not quoted from an earlier document, and the failure claims above were confirmed by
executing them. The framing, the scope boundaries and the decision to keep this record
separate from the matching-architecture one are mine, as was the challenge that overturned
the original address choice (see the discussion below).

[willhea]

Address choice falsified: REJECTED. The record now uses the node ordinal.

Pushed as 26ca34a. The review question was right, and the original choice does not
survive it.

The requirement, stated exactly

Because the key is already scoped by source digest and parser revision, a source change or
a parser change produces a different observation identity by construction. So the address
only has to do this:

Given exact source bytes and exact parser revision, designate one emitted observation,
uniquely and deterministically, and be defined for every emitted observation.

Nothing more. The element id was buying stability across parser changes, which this
contract deliberately does not want.

What the element id satisfies that an ordinal does not

One thing, and it turned out to be partial. Reconstructability from the source without
running the parser: given the XML, a human can find id="H1F2A3" by searching the file,
where an ordinal means nothing without re-running the exact parser revision.

Measured, that property holds for 49,603 of 49,747 nodes. 144 ids, across 48 of the 58
documents, are synthesized by the parser (front-matter-…) and appear nowhere in the
source bytes. So it is strong but not absolute, and it is fully preserved by recording
the element id beside the ordinal rather than keying on it. That is what the record now
does.

Nothing else survived:

property	element id	ordinal
unique within one parse	true on all 129 documents measured, but sampled from a third party's markup, and attrib.get("id", "") already permits an empty one	true because a list index is
defined for every node	measured, not guaranteed	always
deterministic emission needed	no	yes, and this is the new cost
traceable to a source element	99.7%, and kept by recording it	no, and kept by recording the element id
generalizes to the PDF pipeline	no, a block has a page and line but no id	yes, same construct
survives a parser revision bump	yes	no

Why the last row is a hazard rather than a feature

An element id survives a change to how a node's body is extracted, because it is read
from the source element while the body is computed. That is exactly the change that
drifted the answer key: the flagship observation kept its section and its header while its
body went from 81 to 1,443 characters.

A key that still resolves across that change invites auto-migrating a judgment onto a node
that is no longer the same unit. An ordinal fails closed. Under the contract both records
are refused anyway, since the parser revision moved; the difference is which one tempts a
shortcut afterwards.

Evidence

Everything is in scripts/probe_observation_identity.py, which now covers the comparison
as well as the original body-text measurement:

uv run python scripts/probe_observation_identity.py tests/corpus

• Uniqueness of the element id could not be broken. Unique and non-empty on all 58
  committed documents and on a wider 71-document local set. The argument for the ordinal is
  not that the element id is broken; it is that its uniqueness is contingent where the
  ordinal's is not.
• Emission determinism holds. Identical node sequences across repeated parses in one
  process, and a byte-identical corpus digest under three different PYTHONHASHSEED
  values, over both corpus roots. The digest covers content and position.
• The determinism check is proven able to fire. Swapping two adjacent nodes moves the
  ordinal-aware digest and leaves a set-based digest unchanged. Without that, a green
  determinism result would have been indistinguishable from a check that cannot see
  anything.

Invariants that changed

Removed. There is no longer any invariant requiring the element id to be unique or
non-empty, and the synthesized-fallback work for empty ids disappears. The property is
still true everywhere measured; nothing now depends on it staying true. The record says so
explicitly, so its absence reads as deliberate.

Added, and these are the price of the change:

1. Emission is deterministic in content and in order, with the swap injection above as its
   proof of firing.
2. An ordinal indexes the complete emitted sequence, never a filtered or re-sorted
   view. This is a genuine new hazard the element id did not have: indexing a filtered list
   yields an address that looks valid and points at the wrong node.

The first asserts a promise ADR 0008 already makes, rather than adding one.

Two wording corrections, both taken

"The answer key becomes regenerable" overreached and now reads: future and migrated
fixtures become provenance-verifiable and regenerable once the builder and the provenance
fields exist. It states plainly that the existing twelve records cannot gain provenance
that was never stored, and that three still need human legislative adjudication.

The parser revision is no longer over-specified. The architectural requirement is now:

parser_revision is derived from the parser implementation, and changes whenever code
capable of changing the emitted observations changes.

Derived rather than declared is the load-bearing word. The AST content hash is recorded
as an accepted implementation, and swapping it for another mechanism meeting that
requirement needs no amendment.

One thing the change opens rather than closes

The ordinal generalizes to the PDF pipeline where the element id could not, and that was
one reason to prefer it. But determinism has only been measured on the XML side. The record
does not assume the PDF result: it is now an explicit undecided item, with the same test
required before any artifact addresses a PDF block by ordinal.

Also worth flagging

Production already relies on the element id elsewhere. formatters/text_serializer builds
an {element_id: (start, end)} span index that the canonical producer reads, and its
docstring says correctness there "rests on element_ids being present (verified on the
corpus)". That is a within-one-run map rather than a stored key, so a missing id degrades
one report instead of redefining a stored judgment. It is a milder instance of the same
pattern, it is recorded in the consequences, and it is out of scope here.

Gates, re-run on 26ca34a

gate	result
ruff check .	pass
ruff format --check .	pass, 197 files
pytest -m "not slow and not browser"	1666 passed, 5 skipped, 15 xfailed
pytest -m browser	not run (no Playwright browser here; no web surface touched)
pytest -m slow --deselect tests/test_govinfo_corpus_parity.py	exit 0, 1249 passed, 27 skipped, no skip ceiling exceeded