fix: keep the marker and numbers a list was authored with

CHANGELOG.md

@@ -28,6 +28,7 @@ Leafdown uses lightweight [Keep a Changelog](https://keepachangelog.com/en/1.1.0
 
 ### Fixed
 
+- Keep the bullet a list was written with, so a file authored with `-` no longer comes back with its lists rewritten into a mixture of `*` and `-` that follows the order the lists appear in, and a `+` list stays a `+` list. An ordered list keeps its own delimiter and the numbers its items were written with, so `3.` followed by `8.` is no longer renumbered to `3.` and `4.`, and `4)` no longer becomes `4.`. The spaces between a marker and its content are kept too, along with an item whose content was written on the line after its marker. A list made in the editor is still written with `*`, or `.` when it is ordered, and two lists that meet with the same marker are still written apart, because Markdown reads them back as one list.
 - Show a table written with a header row and no body rows as the table it is, instead of adding an empty row beneath it that holds no cells and takes no text.
 - Keep the outer pipes a table's rows were written with, so a table authored without them stays that way instead of gaining one on both sides of every row on the first save. A table inserted from the editor is still written with both, and so is one whose own form would no longer be read back as the table it is.
 - Keep the Markdown an image's description was written with, so `![Alt with *emphasis*](leaf.svg)` keeps its emphasis and `![Outer ![inner](inner.svg)](leaf.svg)` keeps the image inside it, instead of flattening the description to its text on open and losing the inner image's destination from the file on the first save. The image is still named by the text its description spells, and a description edited in the raw image Markdown is written as the text typed there.

docs/specification.md

@@ -258,6 +258,7 @@ For editor input and clipboard ownership, see [Architecture](./architecture.md#e
 - A full, collapsed, or shortcut reference link or image is written back in the form it was authored in, with its definition, rather than as an inline copy of the destination the definition names. Each reference keeps the casing and spacing its label was written with, though references matching one definition still resolve together.
 - A thematic break keeps the character run it was authored with, including the spaces or tabs written between its characters. Indentation before the run and whitespace after it are not part of it and are not written. A break the editor inserts is written as `***`. A break whose authored run would be read back as something else where it lands is written as `***` instead: a run of hyphens directly under a paragraph in a tight list item underlines it, and a run sharing its list item's bullet character joins that bullet into one longer break.
 - A table keeps the outer pipes its rows were authored with, whether both, one, or neither. A table the editor creates is written with both. A table whose rows disagree keeps the pipe the rows that carry one were written with. A table is written with both pipes instead wherever its own form would not be read back: when the first or last cell of any row is blank, or when its first column is one character wide and carries no alignment marker. Cell padding, delimiter row width, and the padding an alignment marker redistributes are normalized rather than kept, because no part of the table owns a width computed across a column.
+- A list keeps the marker it was authored with: `-`, `+`, or `*` for a bullet list, and `.` or `)` for an ordered one. An ordered list keeps the numbers its items were written with rather than renumbering them from its start, except for its first item, which is written with the list's own start because that is the number the file is read back with. Each item keeps the one to four spaces written between its marker and its content, and an item whose content was written on the line after its marker keeps that line. A list the editor creates is written with `*`, or `.` when it is ordered, and an item it creates with one space. Two adjacent lists are never written with the same marker, because CommonMark reads them back as one list: a bullet list moves to `*`, or to `-` where `*` is what collided, and an ordered list moves to the other delimiter. An item is written on its marker's line instead wherever its own form would not be read back: where its list follows a paragraph it has to interrupt, which an item opening on a blank line cannot do.
 - A blank paragraph between blocks survives save and reopen.
 - Save output trims trailing blank lines and writes at most one final line ending, controlled by `Insert final newline on save`. Trailing blank paragraphs go with them.
 

src/features/editor/plugins/commandKeymap.test.ts

@@ -119,7 +119,7 @@ describe("Leafdown editor command keymap", () => {
     const event = dispatchEditorShortcut(mounted.view.dom, "Enter", { ctrl: true });
 
     expect(event.defaultPrevented).toBe(true);
-    expect(mounted.getMarkdown()).toBe("* [x] Task\n");
+    expect(mounted.getMarkdown()).toBe("- [x] Task\n");
   });
 
   it.each([

src/features/editor/plugins/listForm.ts

@@ -0,0 +1,88 @@
+import type { MarkdownNode } from "@milkdown/kit/transformer";
+import { $remark } from "@milkdown/kit/utils";
+
+import {
+  DEFAULT_BULLET_LIST_MARKER,
+  DEFAULT_ORDERED_LIST_MARKER,
+  findListItemForm,
+  LIST_ITEM_LEADING_BLANK_LINE_ATTRIBUTE_NAME,
+  LIST_ITEM_MARKDOWN_TYPE,
+  LIST_ITEM_NUMBER_ATTRIBUTE_NAME,
+  LIST_ITEM_PADDING_ATTRIBUTE_NAME,
+  LIST_MARKDOWN_TYPE,
+  LIST_MARKER_ATTRIBUTE_NAME,
+} from "../utils/listMarkdown";
+
+// Nine digits, a delimiter, and four spaces are the longest head CommonMark reads as a marker and
+// the padding after it.
+const LIST_ITEM_HEAD_LENGTH = 14;
+
+// An item's slice opens at its own marker rather than at the indentation the container gave it, so
+// the head of that slice is the marker and what follows it on the same line.
+const readListItemHead = (item: MarkdownNode, source: string) => {
+  const start = item.position?.start.offset;
+  const end = item.position?.end.offset;
+
+  return start === undefined || end === undefined
+    ? undefined
+    : source.slice(start, Math.min(end, start + LIST_ITEM_HEAD_LENGTH));
+};
+
+// CommonMark puts an item's content one space past its marker wherever the marker's own line
+// carries nothing else, so an item whose first block opens on a later line is one that was written
+// with a blank line after its marker.
+const opensOnLaterLine = (item: MarkdownNode) => {
+  const marker = item.position?.start.line;
+  const content = item.children?.[0]?.position?.start.line;
+
+  return marker !== undefined && content !== undefined && content > marker;
+};
+
+// CommonMark reads a change of marker as the start of another list, so every item of one list was
+// authored with the same one and the first item that carries a position answers for all of them.
+const markAuthoredListForm = (list: MarkdownNode, source: string) => {
+  const ordered = list.ordered === true;
+  let listMarker: string | undefined;
+
+  for (const item of list.children ?? []) {
+    if (item.type !== LIST_ITEM_MARKDOWN_TYPE) {
+      continue;
+    }
+
+    const head = readListItemHead(item, source);
+    const form = head === undefined ? undefined : findListItemForm(head, ordered);
+
+    if (!form) {
+      continue;
+    }
+
+    listMarker ??= form.marker;
+
+    const authored = item as Record<string, unknown>;
+
+    authored[LIST_ITEM_PADDING_ATTRIBUTE_NAME] = form.padding;
+    authored[LIST_ITEM_LEADING_BLANK_LINE_ATTRIBUTE_NAME] = opensOnLaterLine(item);
+
+    if (form.number !== undefined) {
+      authored[LIST_ITEM_NUMBER_ATTRIBUTE_NAME] = form.number;
+    }
+  }
+
+  (list as Record<string, unknown>)[LIST_MARKER_ATTRIBUTE_NAME] =
+    listMarker ?? (ordered ? DEFAULT_ORDERED_LIST_MARKER : DEFAULT_BULLET_LIST_MARKER);
+};
+
+const markAuthoredForm = (node: MarkdownNode, source: string) => {
+  for (const child of node.children ?? []) {
+    if (child.type === LIST_MARKDOWN_TYPE) {
+      markAuthoredListForm(child, source);
+    }
+
+    markAuthoredForm(child, source);
+  }
+};
+
+export const createLeafdownListFormPlugin = () =>
+  $remark("leafdownListForm", () => () => (tree, file) => {
+    markAuthoredForm(tree as MarkdownNode, String(file));
+  });

src/features/editor/plugins/taskListCheckbox.test.ts

@@ -34,7 +34,7 @@ describe("task list checkbox plugin", () => {
 
     expect(event.defaultPrevented).toBe(true);
     expect(mounted.view.dom.querySelector("li[data-checked='true']")).toHaveTextContent("Todo");
-    expect(mounted.getMarkdown()).toContain("* [x] Todo");
+    expect(mounted.getMarkdown()).toContain("- [x] Todo");
   });
 
   it("ignores task-list clicks outside the checkbox hit area", async () => {
@@ -46,7 +46,7 @@ describe("task list checkbox plugin", () => {
 
     expect(event.defaultPrevented).toBe(false);
     expect(mounted.view.dom.querySelector("li[data-checked='false']")).toHaveTextContent("Todo");
-    expect(mounted.getMarkdown()).toContain("* [ ] Todo");
+    expect(mounted.getMarkdown()).toContain("- [ ] Todo");
   });
 
   it("ignores normal list items", async () => {

src/features/editor/tests/keyboardBehavior.test.ts

@@ -32,10 +32,12 @@ describe("Milkdown keyboard behavior", () => {
     setSelectionAtElementTextEnd(mounted.view, listItems[1]);
 
     expect(runKeyDownHandlers(mounted.view, "Tab").handled).toBe(true);
+    // The nested list is one the editor made, so it carries the default marker rather than the
+    // one the list around it was authored with.
     expect(mounted.getMarkdown()).toContain("  * two");
 
     expect(runKeyDownHandlers(mounted.view, "Tab", { shift: true }).handled).toBe(true);
-    expect(mounted.getMarkdown()).toBe("* one\n* two\n");
+    expect(mounted.getMarkdown()).toBe("- one\n- two\n");
   });
 
   it("uses Milkdown defaults to insert hard breaks with Shift+Enter", async () => {

src/features/editor/tests/markdownCompatibility.test.ts

@@ -18,6 +18,8 @@ import {
 import { waitFor } from "@/test/utils/react";
 import { mockTauriApiCommand } from "@/test/utils/tauriApi";
 
+import { runEditorCommand } from "../commands";
+
 const mountEditor = setupMilkdownEditorMount();
 
 const supportedMarkdown = `# Heading
@@ -49,37 +51,8 @@ Footnote[^1]
 
 [^1]: Footnote text`;
 
-// Milkdown serializer defaults normalize several source markers:
-// unordered/task markers become `*`, and serialized output includes a
-// final newline.
-const supportedMarkdownExpected = `# Heading
-
-Paragraph with *emphasis*, **strong**, \`code\`, ~~strike~~, https://example.com, and [link](docs/readme.md).
-
-> Quote
-
-1. One
-2. Two
-
-* A
-* B
-
-\`\`\`ts
-const value = 1;
-\`\`\`
-
----
-
-![Alt](image.png)
-
-${BASIC_TABLE_MARKDOWN}
-
-* [ ] todo
-* [x] done
-
-Footnote[^1]
-
-[^1]: Footnote text
+// The save writes the fixture back as it was authored, apart from the final newline.
+const supportedMarkdownExpected = `${supportedMarkdown}
 `;
 
 const unusualMarkdownFixtures = [
@@ -930,19 +903,30 @@ describe("Thematic break form", () => {
     { name: "a blockquote", source: "> Quote\n>\n> ---" },
     { name: "a list item", source: "* Item\n\n  ---" },
     { name: "a tight list item", source: "* Item\n  ***" },
+    { name: "a list item whose bullet it cannot join", source: "* ---\n  Paragraph" },
   ])("keeps the authored run inside $name", async ({ source }) => {
     const mounted = await mountEditor(`${source}\n`);
 
     expect(mounted.getMarkdown()).toBe(`${source}\n`);
   });
 
   // A bullet and a run spelled with the same character stand on one line and are read back as one
-  // longer break with no list around it, so the run gives way to the default.
-  it("writes a break opening a list item in a run its bullet cannot join", async () => {
-    const mounted = await mountEditor("* ---\n  Paragraph\n");
+  // longer break with no list around it. No file holds that spelling, because it opens as the
+  // longer break rather than as a list, so only a run edited into it can reach the collision. The
+  // run is what gives way, the bullet keeping the marker its list was authored with.
+  it.each([
+    { marker: "---", saved: "- ***", source: "- ***" },
+    { marker: "***", saved: "* ___", source: "* ---" },
+  ])(
+    "writes a $marker break opening $source in a run its bullet cannot join",
+    async ({ marker, saved, source }) => {
+      const mounted = await mountEditor(`${source}\n  Paragraph\n`);
 
-    expect(mounted.getMarkdown()).toBe("- ***\n  Paragraph\n");
-  });
+      setThematicBreakMarker(mounted, marker);
+
+      expect(mounted.getMarkdown()).toBe(`${saved}\n  Paragraph\n`);
+    },
+  );
 
   // A tight list item joins its children with a single newline, so a run of hyphens written after
   // a paragraph there underlines it and the file is read back holding a heading.
@@ -1041,6 +1025,129 @@ describe("Table outer pipe form", () => {
   });
 });
 
+describe("List marker form", () => {
+  const removeBlock = (mounted: MountedMilkdownEditor, index: number) => {
+    const { doc, tr } = mounted.view.state;
+    let start = 0;
+
+    for (let child = 0; child < index; child += 1) {
+      start += doc.child(child).nodeSize;
+    }
+
+    mounted.view.dispatch(tr.delete(start, start + doc.child(index).nodeSize));
+  };
+
+  it.each([
+    "- Hyphen",
+    "+ Plus",
+    "* Asterisk",
+    "1. Period",
+    "1) Parenthesis",
+    // CommonMark reads a change of marker as the start of another list, so adjacent lists were
+    // each authored with a marker of their own.
+    "- Hyphen\n\n+ Plus\n\n* Asterisk",
+    "1. Period\n\n1) Parenthesis",
+    // A nested list is a list of its own and carries its own marker.
+    "- Outer\n  * Nested\n  * Nested again\n- Outer again",
+    "1. Outer\n   + Nested\n2. Outer again",
+    "+ Outer\n  1) Nested",
+    // A task marker stands inside the item rather than in place of its marker.
+    "- [ ] Todo\n- [x] Done",
+    "1) [ ] Todo\n2) [x] Done",
+  ])("writes the marker in %j as it was authored", async (source) => {
+    const mounted = await mountEditor(`${source}\n`);
+
+    expect(mounted.getMarkdown()).toBe(`${source}\n`);
+  });
+
+  // Only an ordered list's first number sets the start it is read back with. The numbers after it
+  // are the author's own counting, which a renumbering from the start would rewrite.
+  it.each([
+    "1. One\n2. Two\n3. Three",
+    "3. Three\n8. Eight\n8. Eight again",
+    "1. One\n1. One again\n1. One more",
+    "0. Zero\n0. Zero again",
+    "123456789. The longest marker CommonMark reads\n1. Short again",
+  ])("writes the numbers in %j as they were authored", async (source) => {
+    const mounted = await mountEditor(`${source}\n`);
+
+    expect(mounted.getMarkdown()).toBe(`${source}\n`);
+  });
+
+  it.each([
+    "- one space",
+    "-  two spaces",
+    "-   three spaces",
+    "-    four spaces",
+    "10.  Padding is measured from the end of the marker",
+    "-    [x] A task marker stands after the padding",
+    // The padding is the column the item's own blocks are written at.
+    "-   Paragraph\n\n    Second paragraph",
+    "-  Paragraph\n   - Nested",
+  ])("writes the marker padding in %j as it was authored", async (source) => {
+    const mounted = await mountEditor(`${source}\n`);
+
+    expect(mounted.getMarkdown()).toBe(`${source}\n`);
+  });
+
+  // A list the editor makes carries no authored marker and writes the default.
+  it("writes a list made in the editor with the default marker", async () => {
+    const mounted = await mountEditor("Paragraph\n");
+
+    await runEditorCommand(mounted.editor, "format.unorderedList");
+
+    expect(mounted.getMarkdown()).toBe("* Paragraph\n");
+  });
+
+  // Two adjacent lists sharing a marker are read back as one list. No file holds that, because a
+  // repeated marker opens one list to begin with, but deleting what stood between two lists does.
+  it.each([
+    { name: "bullet", saved: "- a\n\n* b\n", source: "- a\n\n<!---->\n\n- b\n" },
+    { name: "ordered", saved: "1. a\n\n1) b\n", source: "1. a\n\n<!---->\n\n1. b\n" },
+  ])(
+    "moves the second of two adjacent $name lists off the marker they share",
+    async ({ saved, source }) => {
+      const mounted = await mountEditor(source);
+
+      removeBlock(mounted, 1);
+
+      expect(mounted.getMarkdown()).toBe(saved);
+
+      const reopened = await mountEditor(saved);
+
+      expect(reopened.view.state.doc.childCount).toBe(2);
+    },
+  );
+
+  it.each([
+    "-\n  Content on the line after the marker",
+    "-\n  First\n- Second",
+    "1.\n   Content",
+    "-\n  [x] A task marker opens the content wherever it stands",
+    "-\n  > A block other than a paragraph",
+  ])("writes the item in %j opening on the line after its marker", async (source) => {
+    const mounted = await mountEditor(`${source}\n`);
+
+    expect(mounted.getMarkdown()).toBe(`${source}\n`);
+  });
+
+  // A list interrupts the paragraph above it only where its first item opens with content, so an
+  // item that would open with a blank line there is written on the marker's line instead. Only a
+  // tight item joins a paragraph to the list after it, and no file holds one: a list written there
+  // is read as more of the paragraph, so only an edit that tightens the item reaches this.
+  it("writes an item opening on the line after its marker where its list must interrupt a paragraph", async () => {
+    const mounted = await mountEditor("- Paragraph\n\n  -\n    Nested\n");
+    const position = getEditorNodePosition(mounted, "list_item");
+    const { attrs } = mounted.view.state.doc.nodeAt(position) ?? {};
+
+    mounted.view.dispatch(
+      mounted.view.state.tr.setNodeMarkup(position, undefined, { ...attrs, spread: false }),
+    );
+
+    expect(mounted.getMarkdown()).toBe("- Paragraph\n  - Nested\n");
+  });
+});
+
 describe("Link and image title form", () => {
   it.each([
     '[Double quote](garden.md "Garden")',
@@ -1371,7 +1478,7 @@ describe("Line-final whitespace", () => {
     { expected: "plain\n", initial: "plain", name: "a paragraph, typed twice", typed: "  " },
     { expected: "plain\n", initial: "plain", name: "a paragraph, typed as a tab", typed: "\t" },
     { expected: "# head\n", initial: "# head", name: "a heading", typed: " " },
-    { expected: "* item\n", initial: "- item", name: "a list item", typed: " " },
+    { expected: "- item\n", initial: "- item", name: "a list item", typed: " " },
     { expected: "> quote\n", initial: "> quote", name: "a blockquote", typed: " " },
   ])(
     "converges on $name after a space is typed at its end",
@@ -1465,7 +1572,7 @@ describe("Line-final whitespace", () => {
       name: "a heading",
     },
     {
-      expected: `* item${NO_BREAK_SPACE}\n`,
+      expected: `- item${NO_BREAK_SPACE}\n`,
       initial: `- item${NO_BREAK_SPACE}`,
       name: "a list item",
     },
@@ -1593,7 +1700,7 @@ describe("Line-initial whitespace", () => {
       typed: "\t",
     },
     { anchor: "head", expected: "# head\n", initial: "# head", name: "a heading", typed: " " },
-    { anchor: "item", expected: "* item\n", initial: "- item", name: "a list item", typed: " " },
+    { anchor: "item", expected: "- item\n", initial: "- item", name: "a list item", typed: " " },
     {
       anchor: "quote",
       expected: "> quote\n",
@@ -1736,7 +1843,7 @@ describe("Typed inline mark source", () => {
   });
 
   it.each([
-    { expected: "* item ~~text~~", initial: "- item", name: "a list item" },
+    { expected: "- item ~~text~~", initial: "- item", name: "a list item" },
     { expected: "> quote ~~text~~", initial: "> quote", name: "a blockquote" },
   ])("writes a strikethrough typed in $name", async ({ expected, initial }) => {
     expect(await typeInto(initial, " ~~text~~")).toBe(`${expected}\n`);