Decide the token-stream arrangement mode for regions without block correspondence

[JSv4]

Settles #694: the architecture of DocxDiff's second arrangement mode — the one that renders a
region as a single word-level token stream instead of block-wise — as a design note under
docs/architecture/.

Reading the engine first changed what the decision is

#694 describes the token-stream mode as a set of "narrowly gated constructs" and asks for an
architecture to be decided before anyone implements one. Most of it is already decided and shipping:

• IrCrossParagraphSegmenter.SegmentRegion is already the general region form. Members are a
  region's paragraphs in document order per side, pairs lists the aligner's word-matched pairs, and
  any member not covered by a pair is one-sided at any position — leading, interior or trailing.
  Zero pairs is a legal input.
• IrEditScriptBuilder.TryBuildStoryFinalMixedRegionOp already admits interior zero-pair regions.

What is narrow is the gate, not the mechanism. So the note records the decisions around the
algorithm and cites the segmenter's class remarks for the algorithm itself, rather than restating
them in worse prose where they would go stale.

What it settles

• Placement — a per-region second mode over a region the block aligner delimits, not a
  replacement for block alignment. A wholesale replacement would have to re-derive everything the
  block path gets right, and the evidence base for the block path is not transferable to it.
• Reversibility — written out rather than asserted, from the three construction invariants the
  segmenter's walk already asserts (single-member cells, contiguous slices, a total disjoint cover in
  stream order). The argument is structural, which is the load-bearing part: widening the gate
  cannot break accept ≡ right / reject ≡ left, including paragraph order, because the property
  does not depend on which regions are selected or on how good the matching is.
• Formatting provenance, structure boundaries, cost, moves — each records the contract the code
  already holds (per-cell re-diff through IrTokenDiffer; IsStreamable + HasStructuralCarrier
  exclusions with whole-region maximality; LcsCellCap; the move-source decline) and says which a
  wider gate stresses first. Cost is the obvious one.

What it deliberately leaves open

The selection criterion, and the note says why that is the honest answer rather than a gap.

Two literals stand in for it today, and they are exactly the tuned thresholds #694's point 2 rules
out: the > 8 member cap in the builder, and crossUnitMatches >= 2 || constructPairs > 0 in the
segmenter. Every replacement that can be written from here fails one of the issue's own constraints:

• a coverage fraction is the forbidden threshold — and the segmenter has already been burned
  twice this way; its own pass-1 remark records that "the retain-in-place and yield-to-cross classes
  overlap on coverage, so no threshold separates them";
• "stream every zero-pair region" is the cleanest rule available and may well be right, but it
  contradicts the only evidence in hand — the code comment carrying the size cap says the stream
  constructs were all decoded from small regions while the replace-gap grammar was validated on the
  large ones;
• "emit the whole-story LCS and accept the interleave" assumes Word is token-first at every
  scale, which is DocxDiff: decide the token-stream arrangement mode for regions with no block correspondence #694's premise and is unverified exactly where it matters.

So the criterion has to be decoded, not designed. The note names the measurement that would
settle it — reference compare output for large zero-pair regions (template vs filled-in copy, memo vs
the contract that replaced it, two drafts rewritten rather than edited), scored on whether surviving
fragments stay in place or the region replaces whole — and files it as the first child issue. A
decision about how to decide is what an architecture note owes when the evidence is not in yet.

Test plan

Named in the note, with what a moved pin in each would mean: DocxDiffFuzzRoundTripTests on the wide
sweep (DOCXODUS_FUZZ_SEEDS=2000, since the order-sensitive seeds from #288 are rare),
DocxDiffCorpusBaselineTests, and DocxDiffGapArrangementTests.

Scope

Documentation only — no code, no behaviour change, so no CHANGELOG entry. Cross-referenced from
CLAUDE.md's diff/comparison list and from ir_diff_engine.md, whose stage list did not mention
IrCrossParagraphSegmenter at all.

Child issues are filed and linked below; #694 closes on the decision, not on the children.

Closes #694

🤖 Generated with Claude Code

https://claude.ai/code/session_01SxehJkp2547AyP8LSd6uFS