Skip to content

v0.1.3 — not everything ending in .md is a note

Latest

Choose a tag to compare

@tonydzi tonydzi released this 05 Sep 04:12
· 2 commits to main since this release

Two ways the indexer quietly answered with the wrong text. Both were found on a real synced vault rather than in review, and both now have tests that run with no model download and no network.

Not everything ending in .md is a note. The walk was a plain rglob('*.md'), which descends into dot-directories. On an Obsidian vault synced with Syncthing that pulled in .stversions/ (Syncthing's old revisions), *.sync-conflict-* copies, .obsidian/ plugin cache and .git/ internals.

This is worse than ordinary noise. A stale revision of a note is semantically almost identical to the live one, so it does not sit harmlessly at the bottom of the ranking — it lands right next to the real note in the top-K and answers the question with outdated content. The failure is silent by construction: retrieval looks healthy, the answer is just wrong.

Those paths are now skipped. BRAIN_INDEX_HIDDEN=1 opts back in, because someone will legitimately keep notes in a hidden folder.

A byte-order mark ate the frontmatter. Notes were read as plain utf-8, so a BOM'd file kept a leading U+FEFF and its first line was no longer ---. The frontmatter regex then saw a note with no frontmatter and dropped its date and title without complaint. Reading as utf-8-sig fixes it.

The two behaviours are now iter_notes and read_note, which is what made them testable at all. tests/test_index_notes.py has seven cases over a directory shaped like a real synced vault; they were checked red against the pre-fix behaviour before being called done.

Closes #1 and #2#2 asked for a smoke test that runs without downloading a model, and this is one.