TOON encoder emits quoted notes[] scalar that its decoder rejects, making valid tracked items unreadable

[unbraind]

Summary

pm 2026.7.26 can serialize a string containing the literal text notes[] into a quoted TOON scalar, but the same bundled TOON decoder interprets the bracket segment as an array header and rejects the document. A previously valid tracked item then disappears from pm context/search and makes pm validate fail storage integrity.

This is a core pm/SDK storage round-trip issue, not an extension customization.

Minimal reproduction

Environment:

• @unbrained/pm-cli 2026.7.26
• bundled @toon-format/toon 2.3.1
• Node 22+

import { decode, encode } from "@toon-format/toon";

const encoded = encode({ body: "notes[] unioned; pm validate ok:true" });
console.log(encoded);
// body: "notes[] unioned; pm validate ok:true"

console.log(decode(encoded));
// ToonDecodeError: Line 1: Unexpected content "unioned; pm validate ok"
// between bracket segment and colon

The same failure is visible through pm:

pm get <affected-id> --json
item_document_invalid: TOON item document is not valid TOON

pm validate --json
storage_integrity.status: error
unreadable_item_files: [...]

The affected real document is byte-identical to its committed origin/main version and was produced/accepted by an earlier pm release. The failing line is a quoted body: scalar containing ordinary prose with notes[] followed later by a colon.

Expected

Every string emitted by pm's canonical TOON serializer must round-trip through pm's canonical decoder. Literal bracket syntax inside a quoted scalar must remain scalar content. Existing repository items must remain readable across upgrades.

Actual

The encoder emits a quoted scalar, but the decoder parses notes[] ... : inside that quoted value as structural syntax. pm context, pm search, pm get, and history verification can no longer read the item; pm validate correctly reports it as unreadable.

Suggested acceptance criteria

• Add round-trip regression cases for notes[] unioned; key:value, comments[] ... :, empty brackets, numeric brackets, and ordinary bracketed prose inside every persisted string field.
• Protect pm independently of the upstream dependency version (escape/encode safely or add a narrowly scoped decoder compatibility path).
• Demonstrate serializeItemDocument -> parseItemDocument round trips and an on-disk upgrade fixture from an older pm release.
• Preserve hash/history verification semantics; recovery must not silently rewrite existing item/history data.
• Provide a supported repair/migration path for already-unreadable tracked items.

[coderabbitai]

🔗 Related PRs

#320 - feat(storage): pm_format_version front-matter gating + validate/health checks (pm-ae1u) [merged]
#499 - Harden SDK schema-driven input fidelity and package preflight [merged]
#549 - feat(sdk): ship history time travel, rich reads, and runtime integrity [merged]
#631 - fix(storage): harden integrity scans against unreadable entries [merged]
#712 - SDK context integrity: atomic Plans, batch similarity, sparse settings, and tombstones [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[unbraind]

Additional recovery evidence: the documented supported repair surface cannot recover this state. pm history-redact <id> --literal "notes[] unioned" --replacement "note entries unioned" --dry-run fails before preview with the same item_document_invalid, because it parses the current item first. This leaves no pm-supported non-manual repair path and also causes pm-changelog --all-release-tags to silently omit the unreadable released item from regenerated CHANGELOG output.

[unbraind]

Tracked by the existing all-status canonical lineage: pm-iqgj — strict TOON quoted-scalar bracket recovery, with downstream changelog propagation covered by pm-hybj. Both are closed because their live resolution records the shipped recovery. Current checkout @toon-format/toon 2.3.1 directly round-trips the reported notes[] unioned; pm validate ok:true scalar, so this report is linked without reopening or duplicating work until the exact affected document/runtime path distinguishes the regression from the already-shipped case. The CodeRabbit related-PR inventory is useful context and is acknowledged.

[unbraind]

Root cause found — this is environment drift, not a pm-cli defect. Recommend closing.

Correcting my own report. The distinguishing detail you asked for is the installed @toon-format/toon version, and it was not the one I claimed.

@unbrained/pm-cli 2026.7.26 declares "@toon-format/toon": "^2.3.1" and every affected lockfile here pinned exactly:

"node_modules/@toon-format/toon": {
  "version": "2.3.1",
  "resolved": "https://registry.npmjs.org/@toon-format/toon/-/toon-2.3.1.tgz",
  "integrity": "sha512-sWqEMeAJGMda9qRrfPKrX3C4c33CO9SJuvJzzrcBEn5sbDB+k6pSLkDsfN0rVnLQwR97MV3sDgVbcxQNsq8xVw=="
}

But on disk the resolved package was 2.3.0 in 16 of 18 workspaces, and npm install reported up to date without reconciling it. So pm was loading a decoder that predates the fix, while the lockfile said otherwise.

Confirming the split at the library level (same host, same Node 26):

@toon-format/toon 2.3.0:
  encode({body: "notes[] unioned; pm validate ok:true"})
    -> body: "notes[] unioned; pm validate ok:true"
  decode(...) -> ToonDecodeError: Line 1: Unexpected content "unioned; pm validate ok"
                 between bracket segment and colon

@toon-format/toon 2.3.1: round-trips cleanly.

This exactly reproduces your finding that 2.3.1 round-trips the reported scalar — you were testing the correct version, my environment was not.

Evidence that the version is the whole story

The correlation across 20 tracked workspaces was total. The two workspaces that already had 2.3.1 on disk (pm-brief, pm-web) had zero unreadable items. All 29 unreadable items lived in workspaces pinned to 2.3.1 but running 2.3.0.

Recovery required no content edits, no history rewrite, and no repo change — only rm -rf node_modules/@toon-format/toon && npm install per workspace:

before: 29 unreadable items across 13 of 20 workspaces
after:   0 unreadable items across 20 of 20 workspaces

Previously unreadable documents (e.g. pm-gantt-chart-xcqd, pm-ts-starter-vevn) now pm get normally and are byte-identical to their committed origin/main versions. The documents were never corrupt.

On the scope I reported

The trigger is broader than notes[] — under 2.3.0 it is any bracketed segment followed later in the same quoted scalar by token:, so array[0] index:1 and even a markdown link (markdown [link](url) time: 5) fail, while b:c a[] (colon first) is fine. That breadth is real, but it is entirely a 2.3.0 behaviour and is fixed in the 2.3.1 you already require.

Suggest closing as not-a-defect. The ^2.3.1 floor is correct and sufficient. My earlier comment about pm history-redact being unable to recover stands as accurate but is moot — with the required decoder version there is nothing to recover.

I have filed the one durable improvement this exposed separately: a drifted-runtime install degrades into silently incomplete pm list output rather than a loud failure, which is what made this look like data corruption for hours.

[unbraind]

Tracked in pm-avv3wx. The tracker item records the reproduction, the classification as a storage round-trip property gap rather than a one-off corpus repair, the fence against the two closed TOON items that repaired earlier damage, and the acceptance criteria: a decode-of-encode identity property over adversarial content in CI, a write path that refuses to persist a document it cannot read back, a named-field validate diagnostic, and a one-time sweep of the existing corpus.

[unbraind]

Resolved by the completed tracker item pm-avv3wx and shipped in PR #765. The live PM record includes exact verification evidence and release disposition.