refactor(parsing): consolidate the fence state machine into lines.ts; memory sections via the shared parser#169
Merged
Conversation
… memory sections via the shared parser The dedup follow-up to #168. Behavior-preserving except two Obsidian-parity fixes (both strict upside). - New obsidian-markdown/lines.ts owns the single CommonMark §4.5 fence state machine (advanceFence) plus splitIntoLines (CRLF-safe) and classifyLines. classifyLines moves out of links.ts (it had no link logic); links.ts and the heading parser both thread fence state through advanceFence, so the three former copies collapse to one. - headings.ts: parseHeadings + findTrailingCommentBlockStart drop their two private fence machines and use advanceFence. Parity fix: indented fences (0-3 spaces, CommonMark 4.5) now hide headings, matching link parsing. - memory-store parseSections is rebuilt on the shared parseHeadings (filtered to H1/H2) with a separate entry-count pass. Parity fix: a "## ..."-looking line inside a code fence is no longer mistaken for a memory section. - CRLF: splitIntoLines replaces hand-rolled "\n"-splits at the note-body production sites (vault-filesystem, search-index, vault-patcher, memory-store) so CRLF-authored notes stop leaking a stray CR into heading/section parsing. 915 tests (incl. new lines.ts + parity coverage), build + lint green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
…skip the unused H1 entry-count scan - Rename single-letter callback params in the new tests (h -> heading, o -> outline) per the AGENTS.md explicit-naming convention. - parseSections: compute entryCount only for H2 sections (the sole consumer, listMemoryFiles, discards the H1's) via a single-pass loop — avoiding the whole-file scan plus slice/filter allocations for the H1 title section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Owner
Author
|
✅ Action performedReview finished.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (13)
💤 Files with no reviewable changes (1)
📝 WalkthroughWalkthroughA new ChangesExtract shared Markdown line-parsing primitives into lines.ts
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Jun 22, 2026
aliasunder
added a commit
that referenced
this pull request
Jun 22, 2026
…ort conventions (#172) ## What Capstone to the parser/IO restructure (#168, #169, #171). Adds a **Module layering** section to `AGENTS.md` so the structure those PRs built is durable and enforceable — recorded as rules, not just implied by the file tree. Also lands the structure-polish follow-ups (folder rename + boundary clarifications) agreed during review. It documents: - **What each folder is for** (`obsidian-markdown/` · `vault-operations/` · `mcp-core/` · `search/` · `oauth/` · `utils/`), and that a module's folder follows **what it depends on**, not just its topic. - **Dependency direction** — `obsidian-markdown/` and `utils/` are leaves; `vault-operations/` and `search/` build on them; `mcp-core/` and the wiring depend on everything — plus the "top level is wiring only" rule. - **The `vault-filesystem.ts` ↔ `utils/fs.ts` boundary** — **policy vs. adapter**: `utils/fs.ts` holds only policy-free `node:fs` wrappers, while the atomic-write strategy, path-safety, and the `vaultFs` data API stay in `vault-filesystem.ts`. "Mechanically generic" isn't enough to demote load-bearing vault-I/O policy. - **`utils/` admission** — generic with zero domain knowledge, clearing one of two bars: it consolidates real duplication (multi-consumer), or it is a self-contained, general-purpose primitive (`mapWithConcurrency`). - **Export style by module kind** — service/data-layer modules export a single namespace object (or a factory-closure when stateful: `createSearchIndex`/`createMemoryStore`); pure parsers/helpers use named exports; `links.ts` is the deliberate namespace exception (its `/g` `lastIndex` footgun). ### Structure polish (also in this PR) - Renamed `vault-mcp/auth/` → `vault-mcp/oauth/` so it no longer collides conceptually with the top-level `src/auth.ts` token utilities. Docs + a folder rename — no behavior change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Jul 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Dedup follow-up to #168 — the second half of the parser-layer restructure. Consolidates the duplicated low-level parsing primitives into one home, with two Obsidian-parity fixes as the only behavior changes (both strict upside).
New
obsidian-markdown/lines.tsThe single home for low-level line/fence logic:
advanceFence— the one CommonMark §4.5 fenced-code state machine.splitIntoLines— CRLF-safe line splitting.classifyLines— the{ text, inCode }generator, moved out oflinks.ts(it had zero link logic; it lived there only because link rewriting needed it first).Three fence state machines → one
links.tsandheadings.ts(which had two private copies — insideparseHeadingsandfindTrailingCommentBlockStart) now thread fence state throughadvanceFence. The long-standing docstring claim that this is "the single home of the fence state machine" is finally true.memory-store.parseSections→ sharedparseHeadingsRebuilt on the shared heading parser (filtered to H1/H2) with a separate entry-count pass, replacing a hand-rolled, near-duplicate section parser.
Parity fixes (the only behavior changes)
headings.tspreviously matched fences only at column 0; the shared grammar allows the CommonMark 0–3 space indent, so a# heading-looking line inside an indented code fence is no longer mis-parsed as a heading.## ...-looking line inside a code fence in anAbout Me/file is no longer mistaken for a real section.CRLF unification
splitIntoLinesreplaces hand-rolled"\n"splits at the note-body production sites (vault-filesystem,search-index,vault-patcher,memory-store), so CRLF-authored notes stop leaking a stray\rinto heading/section parsing.Tests
lines.test.tscoveringsplitIntoLines,advanceFence, andclassifyLines(the latter relocated fromlinks.test.ts).Verification
npm run build(tsc, server + cli) ✅npm test— 915 passed ✅npm run lint✅🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Tests
Refactor