fix: D32 layer-2 grounds against the union of source-derived texts

decisions.md

@@ -670,14 +670,26 @@ claiming row — refined 2026-07-18).
 **Decision.** A claim stores both a standalone `claim_text` and a verbatim `source_span` + character
 offsets, plus an `added_context[]` list naming each added substring's bundle source. Acceptance layers,
 cheapest first: (1) deterministic **anchor** — the source span is a real slice of the chunk; (2)
-deterministic **window-membership** — every added substring verbatim-exists in its declared bundle
-source (rejects fabrication); (3) an in-call **entailment self-verdict** (incl. the "*X said* Y entails
-*X said Y*, not *Y*" rule); (4) a **sampled independent** entailment audit (never per-claim). Replaces
-the verbatim-substring gate, which is incompatible with decontextualization. No external knowledge.
+deterministic **window-membership** — every added substring verbatim-exists in the union of
+source-derived bundle elements, while its declared source tag is advisory provenance (rejects
+fabrication, not mislabeling); (3) an in-call **entailment self-verdict** (incl. the "*X said* Y
+entails *X said Y*, not *Y*" rule); (4) a **sampled independent** entailment audit (never per-claim).
+Replaces the verbatim-substring gate, which is incompatible with decontextualization. No external
+knowledge.
 
 **Context.** A decontextualized claim is a rewrite, so it is never a verbatim substring; grounding must
 be provenance + entailment, as every surveyed decompose-then-verify system does. (C6.)
 
+**Amendment (2026-07-29, union grounding).** Layer 2 searches the TARGET CHUNK slice, deterministic
+document header, both available same-section neighbours, and stored context prefix as one
+source-derived membership union. `added_context.source_kind` is still persisted as a best-effort
+pointer but cannot veto a verbatim union match. The #161 GLM-5.2 smoke loss ledger supplied the
+production evidence: 371 of 411 grounding rejections were `added_context_unverified` mislabel
+deaths; sampled correct decontextualizations added names present in TARGET CHUNK turn lines but
+tagged them `header` (258) or `prefix` (99), and only 27 claims from 19 documents survived.
+**Section summaries remain outside the union (the stored prefix, though LLM-derived, is a designed union member per D79's accepted second-order channel); D79 consumption
+rules are unchanged.**
+
 **Refined by D65 (media).** For media-derived documents grounding is **two hops**: the anchor
 (layer 1) proves the claim derives from the *representation* (document.md); it cannot prove the
 ASR heard or the VLM saw correctly. The layer-4 sampled audit therefore becomes

plan/designs/e2_e3_claims_relations_design.md

@@ -119,13 +119,25 @@ the same rule as decontextualization. E2 parses them deterministically into
 `claims.claim_valid_*`; a malformed string falls back to unknown/null for the temporal fields
 without rejecting the claim. Most claims have no stated world-time and leave these null/unknown.
 
+**Amendment (2026-07-29, union grounding):** layer-2 membership is checked against the **union of
+source-derived bundle elements**: the TARGET CHUNK slice, deterministic document header, both
+available same-section neighbours, and the stored context prefix. The model-emitted
+`added_context.source_kind` remains recorded on claims and decision rows as a best-effort provenance
+pointer, but it is advisory: a wrong tag cannot reject text that exists verbatim elsewhere in that
+union. This corrects the GLM-5.2 smoke failure recorded in the #161 loss ledger: 371 of 411 grounding
+rejections were `added_context_unverified` mislabel deaths; sampled claims were correct
+decontextualizations, while names present verbatim in TARGET CHUNK turn lines were tagged `header`
+(258) or `prefix` (99). Only 27 claims from 19 documents survived the old element-local gate.
+**Section summaries and every other LLM-orientation text remain excluded from the union**, so the
+fact-injection defense is unchanged and D79 consumption rules are unchanged.
+
 Acceptance layers four checks, cheapest first:
 
 1. **Anchor** (deterministic): the `source_span` must be a real, in-bounds slice of the target chunk —
    a check the model cannot talk its way past.
-2. **Window-membership** (deterministic): every *added* substring must verbatim-exist in the bundle
-   element it was attributed to. A claim that invents "in San Francisco" with no bundle source is
-   rejected.
+2. **Window-membership** (deterministic): every *added* substring must verbatim-exist somewhere in
+   the source-derived union above. The attribution tag is advisory. A claim that invents "in San
+   Francisco" with no union member containing it is rejected.
 3. **Entailment self-verdict** (in-call, ~free): the model asserts the chunk + bundle entail the
    claim; includes the rule that "*X said* Y" entails "X said Y", not "Y".
 4. **Sampled independent audit** (offline, not per-claim): a separate judge re-checks a sample, because
@@ -169,7 +181,7 @@ lands a claim is indistinguishable from a keep that produced a claim the gate re
 | `decision_type` | When written | `source_span` | `edit_detail` |
 |---|---|---|---|
 | `claimify_omitted` | A kept Selection span for which Claimify returned **no claim at all** (the model simply skipped it). One row per dead keep. | The Selection span. | null |
-| `grounding_rejected` | A Claimify-returned claim rejected by a D32 gate. One row per rejected claim. | The claim's returned `source_span` (even if not findable in the chunk). | `{"gate": "span_not_found" \| "outside_kept_ranges" \| "added_context_unverified", "claim_span": <truncated>}`; for `added_context_unverified` also `{"kind": ..., "text": <truncated>}`. |
+| `grounding_rejected` | A Claimify-returned claim rejected by a D32 gate. One row per rejected claim. | The claim's returned `source_span` (even if not findable in the chunk). | `{"gate": "span_not_found" \| "outside_kept_ranges" \| "added_context_unverified", "claim_span": <truncated>}`; for `added_context_unverified` also `{"kind": ..., "text": <truncated>, "searched_elements": [...]}`. |
 
 **Invariant — every kept span is accounted for end-to-end.** Two independent rules (revised
 2026-07-27 after review — one keep can decompose into several returned claims with mixed fates,

src/rememberstack/workers/e1.py

@@ -58,14 +58,15 @@
 07c adds D79's bounded current-generation section-summary orientation."""
 
 E2_EXTRACTOR_VERSION: Final = (
-    f"e2-extract-2026.07f:loss-ledger-1:{SECTION_ORIENTATION_VERSION}"
+    f"e2-extract-2026.07g:union-grounding-1:{SECTION_ORIENTATION_VERSION}"
 )
 """The extractor generation baked into extraction_input_hash (D56); the E2
-stage (WP-1.3) binds its handler to this same constant. 07f adds D79 summary
-orientation to the bundle without making summaries hash or grounding inputs;
-07e ledgers Claimify omissions and grounding-gate rejections on the D33
-transcript (#161); 07d pinned temperature=0.0 on the Selection call (Claimify
-already carried it)."""
+stage (WP-1.3) binds its handler to this same constant. 07g makes D32 layer-2
+grounding union-based across source-derived bundle texts with advisory source
+tags; 07f adds D79 summary orientation to the bundle without making summaries
+hash or grounding inputs; 07e ledgers Claimify omissions and grounding-gate
+rejections on the D33 transcript (#161); 07d pinned temperature=0.0 on the
+Selection call (Claimify already carried it)."""
 
 _PREFIX_PROMPT_TEMPLATE: Final = (
     "In one sentence, state where this passage sits in the document — "

src/rememberstack/workers/e2.py

@@ -4,10 +4,11 @@
 drop — drops and flags go to the D33 ledger), then one fused call
 decontextualizes, decomposes, and self-grounds the keeps. The deterministic
 grounding gate (D32 layers 1-2) accepts a claim only if its verbatim source
-span anchors inside the chunk and every added substring exists in the bundle
-element it was attributed to — a check the model cannot talk its way past.
-Every kept span ends in accepted claim(s), grounding_rejected row(s), or a
-claimify_omitted row so Claimify-stage losses are never silent (#161).
+span anchors inside the chunk and every added substring exists in the union of
+the bundle's source-derived texts. The model's source tag is advisory provenance,
+not an acceptance boundary. Every kept span ends in accepted claim(s),
+grounding_rejected row(s), or a claimify_omitted row so Claimify-stage losses
+are never silent (#161).
 """
 
 from dataclasses import dataclass
@@ -86,16 +87,20 @@
 careful reader could not pick one interpretation from the bundle, omit the
 candidate. For each claim return: claim_text (standalone), source_span (the
 verbatim chunk substring it derives from), added_context (every substring you
-ADDED, each tagged header|neighbour|prefix with the exact text as it appears
-in that bundle element; SECTION SUMMARIES are orientation only, never quotable
-and never an added_context source), entailment_self_verdict (does chunk+bundle
-entail the claim), is_attributed. When the source states or implies WHEN a fact
-holds or happened, resolve relative dates USING ONLY THE BUNDLE (as with
-decontextualization) and emit valid_kind, valid_from_iso, valid_until_iso, and
-valid_precision. Use ISO-8601 dates (YYYY-MM-DD) or datetimes WITH an explicit
-offset or Z; never emit a datetime without an offset. Otherwise leave
-valid_kind/from/until null and valid_precision unknown. Event on a calendar day
-→ valid_kind=event_time, valid_precision=day, both ISO ends for that day.
+ADDED that is not already present in the TARGET CHUNK; in-chunk text needs no
+added_context entry). Tag each addition header|neighbour|prefix as a best-effort
+provenance pointer, but the tag is advisory: every addition must exist verbatim
+somewhere in the bundle's source-derived texts (TARGET CHUNK, DOCUMENT HEADER,
+same-section PREVIOUS/NEXT CHUNK, or stored CONTEXT PREFIX). SECTION SUMMARIES
+are orientation only, never quotable and never an added_context source. Also
+return entailment_self_verdict (does chunk+bundle entail the claim) and
+is_attributed. When the source states or implies WHEN a fact holds or happened,
+resolve relative dates USING ONLY THE BUNDLE (as with decontextualization) and
+emit valid_kind, valid_from_iso, valid_until_iso, and valid_precision. Use
+ISO-8601 dates (YYYY-MM-DD) or datetimes WITH an explicit offset or Z; never
+emit a datetime without an offset. Otherwise leave valid_kind/from/until null
+and valid_precision unknown. Event on a calendar day →
+valid_kind=event_time, valid_precision=day, both ISO ends for that day.
 Year-only → precision=year with that year's [start,end] ISO bounds; quarters
 are calendar quarters. Bounded
 precisions (day|month|quarter|year) require both ends; open requires from only;
@@ -352,6 +357,7 @@ class GroundingRejection:
     claim_span: str
     kind: str | None = None
     text: str | None = None
+    searched_elements: tuple[str, ...] = ()
 
 
 def _grounded_claim(
@@ -370,12 +376,15 @@ def _grounded_claim(
     Layer 1 (anchor): the source span must be a real in-bounds slice of the
     target chunk, and must overlap a span Selection kept — the fused call can
     never resurrect a dropped proposition. Layer 2 (window membership): every
-    added substring must verbatim-exist in the bundle element it was
-    attributed to. A failed check returns which gate fired so the D33 ledger
-    can record ``grounding_rejected`` (#161); the accept path is unchanged.
-    Semantic invention behind a real span is layer-3/4 territory: the in-call
-    self-verdict is stored advisory, and the sampled independent audit owns
-    the honest measurement.
+    added substring must verbatim-exist in the union of source-derived bundle
+    texts. The model's ``source_kind`` is preserved as advisory provenance but
+    cannot reject a grounded addition by being wrong. Section summaries are
+    excluded from this union (the stored prefix, though LLM text, is a
+    designed union member — D79's accepted second-order channel). A failed check
+    returns which gate fired so the D33 ledger can record
+    ``grounding_rejected`` (#161). Semantic invention behind a real span is
+    layer-3/4 territory: the in-call self-verdict is stored advisory, and the
+    sampled independent audit owns the honest measurement.
     """
     claim_span = candidate.source_span
     anchor_at = document_md.find(claim_span, chunk.char_start, chunk.char_end)
@@ -392,20 +401,17 @@ def _grounded_claim(
         return GroundingRejection(
             gate=GroundingGate.OUTSIDE_KEPT_RANGES, claim_span=claim_span
         )
+    grounding_elements = _source_grounding_elements(
+        source=source, chunks=chunks, index=index, document_md=document_md
+    )
     for added in candidate.added_context:
-        element = _bundle_element(
-            kind=added.source_kind,
-            source=source,
-            chunks=chunks,
-            index=index,
-            document_md=document_md,
-        )
-        if element is None or added.text not in element:
+        if not any(added.text in text for _, text in grounding_elements):
             return GroundingRejection(
                 gate=GroundingGate.ADDED_CONTEXT_UNVERIFIED,
                 claim_span=claim_span,
                 kind=added.source_kind,
                 text=added.text,
+                searched_elements=tuple(name for name, _ in grounding_elements),
             )
     valid_from, valid_until, valid_precision, valid_kind = _parse_claim_valid_time(
         candidate=candidate
@@ -567,30 +573,40 @@ def _bundle_text(
     )
 
 
-def _bundle_element(
+def _source_grounding_elements(
     *,
-    kind: str,
     source: ChunkSource,
     chunks: tuple[ChunkForEmbedding, ...],
     index: int,
     document_md: str,
-) -> str | None:
-    """The bundle element an added substring claims to come from, or None."""
-    if kind == "header":
-        return _header_text(source=source)
-    if kind == "prefix":
-        return chunks[index].context_prefix
-    if kind == "neighbour":
-        return "\n".join(
-            _neighbour_text(
-                chunks=chunks,
-                index=neighbour,
-                document_md=document_md,
-                section_path=chunks[index].section_path,
+) -> tuple[tuple[str, str], ...]:
+    """Return the complete D32 layer-2 membership union.
+
+    Every member is source-derived: the target chunk slice, deterministic
+    document header, available same-section neighbours, and the stored context
+    prefix. Section summaries are deliberately absent; D79 orientation text
+    must never become a fact-injection path.
+    """
+    chunk = chunks[index]
+    elements = [
+        ("target_chunk", document_md[chunk.char_start : chunk.char_end]),
+        ("document_header", _header_text(source=source)),
+    ]
+    for name, neighbour_index in (
+        ("previous_same_section_neighbour", index - 1),
+        ("next_same_section_neighbour", index + 1),
+    ):
+        if (
+            0 <= neighbour_index < len(chunks)
+            and chunks[neighbour_index].section_path == chunk.section_path
+        ):
+            neighbour = chunks[neighbour_index]
+            elements.append(
+                (name, document_md[neighbour.char_start : neighbour.char_end])
             )
-            for neighbour in (index - 1, index + 1)
-        )
-    return None
+    if chunk.context_prefix:
+        elements.append(("context_prefix", chunk.context_prefix))
+    return tuple(elements)
 
 
 def _header_text(*, source: ChunkSource) -> str:
@@ -725,6 +741,7 @@ def _grounding_rejected_decision(
         # kind is model-returned text too — bound every persisted field
         edit_detail["kind"] = _truncate_for_ledger(rejection.kind or "")
         edit_detail["text"] = _truncate_for_ledger(rejection.text or "")
+        edit_detail["searched_elements"] = list(rejection.searched_elements)
     return DecisionRecord(
         decision_id=uuid4(),
         deployment_id=source.deployment_id,