fix(editor): restore inline date pills on the collaborative open path; legible fired reminders - #1866
Merged
Conversation
An inline reminder pill turned into its own base64 after a restart or a vault switch. Main parses the vault file into the shared Y.Doc with a `dateMention` spec whose `parse` claims a `data-date-mention` element, which markdown has none of, so the token reaches the doc as plain text -- and `useEditorSync`'s load effect returns before `normalizeNoteBlocks` on that path, so nothing promoted it back. Wiki links and inline checkboxes already close the same gap; date mentions now do too. Byte-neutral for a well-formed token, so a second open writes no CRDT update. The token alphabet is closed at the same time. base64url's `_` opens emphasis and sits on remark-stringify's unconditional escape list for phrasing content, so the encoder now spells the two odd base64 symbols `,` and `;`, which no markdown writer escapes. Real payloads are printable-ASCII JSON and never reach those symbols, so every token already on disk re-encodes to identical bytes and no note is rewritten. The parser accepts both alphabets and strips a stray `\` escape. A token the parser refuses now degrades to a plain, readable date instead of staying an opaque blob. Bytes with no recoverable date are left alone. Fired reminder pills read as a spent chip -- muted text at full opacity, a settled background, a dimmed alarm -- rather than as an error in red. Closes #1845
|
React Doctor found no new issues. 🎉 Reviewed by React Doctor for commit |
…replace CodeQL reads a `replace` that prefixes a backslash as an escaping routine and files js/incomplete-sanitization against it. These are fixtures that deliberately corrupt a base64url run, not sanitizers, and a named helper says so.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
h4yfans
added a commit
that referenced
this pull request
Aug 27, 2026
#1866 gave `((date:…))` a promoter on the collaborative path, so the list of nodes that reach the shared doc as text and get rebuilt there is one longer. Names hash tags as the one that stays text, and why.
h4yfans
added a commit
that referenced
this pull request
Aug 27, 2026
The hostile anchor bytes that forced base64url emphasis runs now assert the opposite property: the fixed encoder emits no emphasis character at all, so a future alphabet reopen fails the corpus loudly.
h4yfans
added a commit
that referenced
this pull request
Aug 27, 2026
#1866 gave `((date:…))` a promoter on the collaborative path, so the list of nodes that reach the shared doc as text and get rebuilt there is one longer. Names hash tags as the one that stays text, and why.
h4yfans
added a commit
that referenced
this pull request
Aug 27, 2026
The hostile anchor bytes that forced base64url emphasis runs now assert the opposite property: the fixed encoder emits no emphasis character at all, so a future alphabet reopen fails the corpus loudly.
h4yfans
added a commit
that referenced
this pull request
Aug 28, 2026
The hostile anchor bytes that forced base64url emphasis runs now assert the opposite property: the fixed encoder emits no emphasis character at all, so a future alphabet reopen fails the corpus loudly.
h4yfans
added a commit
that referenced
this pull request
Aug 28, 2026
#1866 gave `((date:…))` a promoter on the collaborative path, so the list of nodes that reach the shared doc as text and get rebuilt there is one longer. Names hash tags as the one that stays text, and why.
h4yfans
added a commit
that referenced
this pull request
Aug 28, 2026
…e reload (#1867) * fix(editor): close the link-mention token alphabet so mentions survive reload A link mention persists as literal `((mention:<encoded-url>))` text inside a paragraph, so it has to survive both the markdown writer and the markdown reader. Two separate holes meant it did not. The collaborative load path never rebuilt the chip. Main seeds the shared Y.Doc straight from the vault file, where the mention is plain text, and the Yjs branch of `useEditorSync` returns before `normalizeNoteBlocks` runs. Wiki links and inline checkboxes each have a promoter on that path; mentions had none. A mention therefore lived only as long as its Y.Doc and came back as literal text after a restart or a vault switch. The token's alphabet was open. `encodeURIComponent` leaves `- _ . ! ~ * ' ( )` raw, and remark-stringify does not escape them inside the token — so two mentions on one line whose URLs each hold a `*` (or each a `~`) are read back as a single emphasis or strikethrough run spanning from the first token into the second, and both mentions are destroyed. Encoding `! ' ( ) * ~ _` closes the payload to `[A-Za-z0-9.%-]`, which nothing downstream can reinterpret. `MENTION_TOKEN_REGEX` widens from `[^)\s]+` to `[^)\n\r]+` — a strict superset, so today's tokens match with identical captures — and `parseLinkMentionToken` keeps its original code path for any payload without whitespace or a backslash, adding a repair path that strips those and requires a parseable URL before accepting the result. Tokens already mangled on disk heal on open; tokens that are fine keep their exact bytes. * docs(architecture): note the collaborative-path promoters and the mention token alphabet * fix(editor): guard the mention pattern against a broken token upstream Review of the widened payload class turned up a case where it loses a mention the strict pattern found. Stopping only at `)` runs the match straight through an unterminated `((mention:` fragment earlier on the line and captures it together with the next real token, so a note damaged by this very bug — which is where those fragments come from — would lose a mention that works today: x ((mention:foo ((mention:https%3A%2F%2Fx.test)) y strict -> https%3A%2F%2Fx.test stop-at-paren -> foo ((mention:https%3A%2F%2Fx.test Excluding `(` restores the strict pattern's captures. No serializer has ever emitted a raw paren in a payload, so nothing real is refused. The repair path now also requires what it repaired to be spellable in the token alphabet. `new URL()` alone was not the gate the comment claimed: with the spaces stripped out, `((mention: see http://x.com))` parses as a URL and became a chip pointing at `seehttp://x.com`. Favicon hydration is dropped from the collaborative path. It writes back a content array captured before its fetch, which on a shared document pushes a stale block to every device, and it resolves after `clearYjsUndoHistory`, so opening a note left an undoable step — the invariant the surrounding code exists to hold. A chip promoted there renders the domain the token carries. Hydrating a shared document safely is its own change. * docs(notes): say what a link mention shows after a restart * docs(architecture): fold the date pill into the promoter list #1866 gave `((date:…))` a promoter on the collaborative path, so the list of nodes that reach the shared doc as text and get rebuilt there is one longer. Names hash tags as the one that stays text, and why.
h4yfans
added a commit
that referenced
this pull request
Aug 28, 2026
The hostile anchor bytes that forced base64url emphasis runs now assert the opposite property: the fixed encoder emits no emphasis character at all, so a future alphabet reopen fails the corpus loudly.
h4yfans
added a commit
that referenced
this pull request
Aug 28, 2026
…zers (#1874) * test(editor): golden round-trip conformance suite across both serializers One corpus in @memry/editor-schema/conformance, asserted byte-for-byte by both pipelines: the renderer save path (real BlockNoteEditor + the real editor schema, markdown-utils + normalize-note-blocks) and the main/CRDT converter (markdownToYFragment -> yDocToMarkdown). Each case asserts round-trip identity and second-pass idempotence against the same canonical bytes, so cross-serializer agreement is a tested equality. Covers the payload table of #1848: mention URLs with markdown-significant characters, date tokens with base64url emphasis runs, all four callout types plus foreign passthrough / titles / multi-paragraph bodies, toggles empty / nested / blank-lined / fenced / imaged / unterminated, and tokens inside table cells, list items and toggle bodies — plus a deterministic seeded fuzz layer over the token alphabets (no property-testing dep in the workspace). Cases broken on current main are marked it.fails pending their sibling fix (#1844 #1846 #1847, plus #1865 found by this suite), so the sibling landing turns the inverted expectation red and forces the flag off. Golden vault gains four roundtrip-*.md fixtures, byte-gated by the existing frontmatter suite and now also pushed through the converter. New renderer-side counter reports tokens the normalize chain left unclaimed through app_error_seen (leading emit, then one aggregated event per kind per minute). Metric only, no user-facing surface. * docs(observability): document the unclaimed-token counter * test(editor): keep pending buckets valid once every sibling fix lands it.each on an empty list throws, so the it.fails buckets are registered only while pending cases exist; conformance internals stop being exported. * test(editor): deepen the fuzz corpus with whole-document mixing New mixed-documents family assembles notes from every token family plus lists, paragraphs and gaps. It immediately caught #1877 — blank-line gaps adjacent to a toggle collapse on write-back — now pinned as a static pending case; the family itself joins with single gaps and never puts two lists across one blank line (CommonMark fuses those: accepted canonicalization, not token damage). * test(telemetry): cover string-shaped inline content in the token walk * test(editor): adapt the date corpus to the closed token alphabet (#1866) The hostile anchor bytes that forced base64url emphasis runs now assert the opposite property: the fixed encoder emits no emphasis character at all, so a future alphabet reopen fails the corpus loudly. * test(editor): assert the fixed callout behavior landed by #1875 Foreign passthrough, titles after the marker and the four claimed types now round-trip on both pipelines as plain assertions. The two blank-`>`- line shapes #1875 explicitly declined re-anchor to #1881: the quote path collapses blank quote lines on ANY blockquote, callout or not, merging quoted paragraphs on write-back. * test(editor): assert persisted toggle fold state landed by #1876 Expanded toggles (the new open-attribute on-disk form) join the corpus, the golden fixture and the fuzz roll. The unterminated shape #1876 did not reach re-anchors to #1883: the declined region's raw-HTML lines are still dropped by the markdown parser on both pipelines. * test(editor): assert the closed mention alphabet landed by #1867 Every mention payload and the mention fuzz family now assert fixed behavior on both pipelines. The renderer's rich render-in-a-cell rewrite was not in #1867's scope and joins its wiki-link twin on #1865.
8 tasks
h4yfans
added a commit
that referenced
this pull request
Aug 28, 2026
The spec still asserted the pre-#1866 triggered red (#e56458). PR #1866 deliberately restyled fired pills to muted-foreground with a settled fill, and E2E only runs on push to main, so the stale assertion first surfaced there. Assert the new contract: muted color plus a background fill that a date-only pill does not have.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes for the user
An inline reminder pill written into note body text survives a restart, a vault switch or a rebuild on a new device. Before this, the pill came back as a two-hundred-character base64 run — the date it stood for was unreadable, which is what the reporter hit.
A fired reminder pill now reads as a spent chip: muted text at full opacity, a settled background, a dimmed alarm icon. It keeps its date and time on the page as a record of what was scheduled, and is visually distinct from an armed pill (blue, alarm) and from a date-only pill (muted, no fill, no alarm). Red is left to mean broken.
A date token the parser refuses now degrades to a plain, readable date instead of staying an opaque blob.
Root cause
Not the escape the issue proposed. Two probes against the real converter say so: a realistic payload never reaches the base64 symbols base64url spells
-/_(the JSON is printable ASCII, and only>,?or~can drive base64 onto them — no field can hold one), and when a token is forced to contain_, this serialization path emits it bare rather than escaped.The actual break is the collaborative open path. Main parses the vault file into the shared Y.Doc with a
dateMentionspec whoseparseclaims adata-date-mentionelement, and markdown has none, so the token reaches the doc as plain text.useEditorSync's load effect returns early once a Y.Doc fragment is bound, sonormalizeNoteBlocks— and every promoter in it — runs on the markdown path only. Nothing turned the token back into a pill. Wiki links (#1642) and inline checkboxes already close this exact gap; a date token is worse off than either, because its text carries no readable fallback at all.promoteDateMentionsInSharedDoccloses it the same way.Compat plan
,and;rather than base64url's-and_, because_opens emphasis and sits on remark-stringify's unconditional escape list for phrasing content. No realistic payload reaches those symbols, so every token already on disk re-encodes to identical bytes.date-mention.test.tsasserts byte-for-byte equality against the old writer over 200 generated payloads. The write-back byte compare therefore rewrites nothing.\inside the run is stripped, so any token already written with an escape is healed on read.parseDateMentionTokenstill returns null for pre-remind-enum tokens. The degrade is a separatesalvageDateMentionToken, and it restoresremind: 'none'only — those tokens have armed nothing since the enum landed, so nothing is resurrected.data-firedDOM attribute byuse-triggered-date-pills.tsand is never written into pill props or markdown.Verification
pnpm lintclean.typecheck:node/typecheck:web/typecheck:testclean.pnpm typecheckfails only on a pre-existing architecture-boundary violation inapps/mobile/vitest.config.ts, which this branch does not touch.packages/shared/src/date-mention.test.ts— 22 passed. Covers the closed alphabet, byte identity against the old writer, a legacy base64url token with_/-runs, a backslash-escaped token, and the salvage.apps/desktop/src/main/sync/blocknote-converter.test.ts— 247 passed withsrc/main/notes/note-date-reminders.test.ts. New: adateMentionnode serializes to exactly the token the renderer's promoter reads, emits only[A-Za-z0-9,;], hands an un-promotable token back byte-identical, and drops a stray escape rather than carrying it.date-mention-collab-promotion.test.ts— 6 passed. RealBlockNoteEditoron the real schema, real Yjs collaboration, the real hook. Covers the current token, the legacy one, the escaped one, the degrade, the untouched-bytes case, and "promotes once, writes nothing on a second open".pnpm docs:impact --base origin/main --strictgreen,pnpm docs:buildgreen,git diff --checkclean.The full conformance suite is #1848's; the link-mention half is #1844's, and
link-mention.tsis untouched here.Closes #1845