Skip to content

review/content-lane: duplicates.ts's block-scalar detection lags source-evidence.ts's hardened version (comment-tolerance, digit order) #9290

Description

@JSONbored

⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.

Context

src/review/content-lane/duplicates.ts has two call sites (parseSimpleFrontmatter at line ~77
and findDuplicateFrontmatterKeys at line ~132) that detect a YAML block-scalar header (|, >,
|-, |2-, etc.) using:

/^[|>][+-]?\d*$/.test(inline)

The sibling parser in src/review/content-lane/source-evidence.ts was previously hardened (in
#8016) into a shared, more correct check:

const BLOCK_SCALAR_INDICATOR = /^[|>](?:[+-]?\d?|\d[+-]?)$/;
function isBlockScalarHeader(raw: string): boolean {
  return BLOCK_SCALAR_INDICATOR.test(stripYamlComment(raw));
}

This hardened version (a) accepts the indentation digit before or after the chomping indicator
(|2- as well as |-2), and (b) strips a trailing inline comment first, so | # sources below
is still recognized as a block-scalar header. duplicates.ts's regex has neither fix.

Concretely, duplicates.ts fails to recognize: |2- (digit-then-chomp order), and any
block-scalar header with a trailing comment (| # note, |- # note, |2- # trailing). When that
happens, the header line collapses to the literal string "|" and the real indented block content
that follows is silently skipped by the parser loop (indented lines don't match the key: regex).

This directly undermines duplicates.ts's own stated purpose (per its file-level doc comment: "a
regex parser that silently drops block/folded/list values would let a contributor hide a
protected-field edit and bypass the protected-edit + duplicate gates"). A protected field written
as description: | # note followed by indented content would parse to "|" both before and after
an edit, so protectedFrontmatterChanges would never detect the real content change, and two
semantically-different entries using this style would collide as "same normalized description" in
the duplicate detector.

This is the same cross-file divergence class already fixed twice between these exact two files:
#7250 and #8016 (both closed) — this issue is the same class of gap, this time in the
opposite direction (source-evidence.ts is ahead, duplicates.ts lags).

Requirements

  • duplicates.ts's two block-scalar-header checks (in parseSimpleFrontmatter and
    findDuplicateFrontmatterKeys) must recognize a block-scalar header exactly as
    source-evidence.ts's isBlockScalarHeader/BLOCK_SCALAR_INDICATOR/stripYamlComment already
    do — including the digit-before-or-after-chomp order and trailing-comment tolerance.
  • Prefer factoring this into one shared helper both files import (avoiding a third
    independently-drifting copy), but a direct port of the three pieces
    (BLOCK_SCALAR_INDICATOR, isBlockScalarHeader, stripYamlComment) into duplicates.ts is
    also acceptable if a shared module isn't a clean fit — pick one, don't leave a partial mix.
  • Do not change source-evidence.ts — it is already correct and is the precedent to copy from.
  • Do not change the overall parsing behavior for non-block-scalar values.

Deliverables

  • duplicates.ts's parseSimpleFrontmatter recognizes |2--style and comment-trailing
    block-scalar headers identically to source-evidence.ts.
  • duplicates.ts's findDuplicateFrontmatterKeys recognizes the same headers identically.
  • New test cases (in the existing duplicates.ts test file under test/unit/) covering: a
    |2- header, a |- # comment header, and a > # comment header, each asserting the
    indented block content that follows is correctly captured (for parseSimpleFrontmatter) or
    correctly skipped without producing a spurious duplicate/parse error (for
    findDuplicateFrontmatterKeys).

All three deliverables are required in this single PR.

Test Coverage Requirements

This repo enforces 99%+ Codecov patch coverage, branch-counted, on every changed line/branch in
src/**. The new test cases above must exercise both fixed call sites.

Expected Outcome

duplicates.ts recognizes the same set of block-scalar header forms source-evidence.ts already
does, so a protected frontmatter field written with a trailing comment or digit-before-chomp
indentation marker is no longer silently collapsed to "|", closing the protected-edit/duplicate
detection bypass described above.

Links & Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions