Skip to content

v10.3.0 — Honest tools

Choose a tag to compare

@miisodev miisodev released this 16 Aug 16:26
· 15 commits to main since this release

V10.2 made the project reachable. This release makes its tools honest — it closes a register of 21 defects and requests accumulated across two weeks of daily use, and the through-line is that almost every one of them reported success while being wrong.

Three of them shipped once with a passing test suite, because the suite tested what the code did rather than what it was for. Each now carries a regression test asserting the failure mode.

The corruption that kept coming back

escapeHtml() was a blanket && with no entity guard, and it runs on every paragraph, heading, list item and table cell taking the markdown path. Any plain body already carrying an entity — an   copied out of a prior read, a stray <, a URL with an ampersand — was escaped a second time and stored as visible literal text.

This was declared fixed in August and recurred twice. The reason is precise and worth stating: the August work hardened looksLikeEncodedHtml() and decodeEncodedHtml(), which are the detector and the repairer. The producer sat one function away and was never touched. The guard now applied is the same one decodeEncodedHtml() was already using.

maintain() had the matching blind spot — it searched only for the literal <, while the commonest corruption this bug produces is  . It now matches any doubly-escaped entity, and maintain(repair=[noteId]) unwinds one level in place with a revision taken first.

Backslashes were being deleted before the query ran

consistency() and recall(regex=) routed their patterns through escapeQueryValue(), which replaces every backslash with a space. So (\d+) migrations reached the backend as ( d+) migrations and matched nothing, while ([0-9]+) found the real answer.

A zero-result contradiction sweep reads as "the brain agrees" — the exact opposite of the truth. This was a silent wrong answer, not a failed call.

Trilium is not at fault: %= compiles the string with new RegExp(str, "ms") and supports \d fine. Its lexer is, and the rule is now measured rather than inferred — running Trilium's own lex() gives backslash survival of floor(n/2), identically inside single and double quotes, with \s \b \. \w all consumed at one backslash. escapeQueryRegex() therefore doubles, and the full chain is verified end to end.

One query, two different documents

Trilium runs %= against content its preprocessor has already put through striptags; both tools then re-ran the regex against the raw stored body. Two different corpora behind one query, failing in both directions: a phrase split by an inline <strong> or <code> was found by the backend and silently dropped locally, and a pattern anchored on tags never entered the candidate set at all.

Both tools now match the raw body and a tag-stripped projection, and report how many hits existed only after stripping. consistency() scans exhaustively by default — the %= pre-filter is lossy in both directions, and on a contradiction sweep a falsely clean result is worse than a slow one.

The read side finally targets headings

Every write tool could target a heading and no read tool could, so seeing one section meant reading the whole note — worst on exactly the notes where it hurts most. A Current State note grew 53k → 108k characters across five sessions and at one point could not be returned at all: the notes carrying the most were the ones the brain could least afford to open.

section= now works on knowledge, memory, llm, master, insights and inspect, added once to the shared read helper. More importantly, setSection() and the new getSection() go through one matching contract — so a heading name that reads also writes, structurally rather than by keeping three copies of a comparison in step.

Also on the read side: diff(noteId) compares a note against the revision your last write snapshotted (every write already took one; nothing could read one back), mode="remove" deletes a section — which previously had no working path at all, since the whole section machinery sat behind an if (body) guard — and a find= miss now returns the longest fragment that is present plus the stored text around it, instead of only confirming absence.

Checking the brain against the world

consistency() asks whether the brain agrees with itself. Nothing asked whether an assertion is still true of the codebase or live surface it describes, so a claim that quietly stopped being true stayed authoritative until something downstream broke — twice nearly shipping a wrong answer.

claim() registers a checkable assertion, records verifications with evidence, and lets maintain(deep=true) surface the lapsed and the broken. BrainLLM never executes the check: it has Trilium access and no shell, and content inside a note is data rather than instructions — a recipe that ran itself would be an injection surface pointed at the user's machine, reachable by anything able to write a note. The tool owns the register and the schedule; the agent runs the check and reports back.

Also

brain() rows carry parent — read from data already present on every result, and absent only because nothing read it. That absence caused real loss: each group is a flat list whose ordering looks nested, an audit concluded a domain held only a Sources note, and the replacement it wrote deduped by title over an 8,259-byte note.

remember(mustCreate=true) refuses to adopt an existing note instead of silently updating it. Current State, Sources, Technology Stack and Product and Business each exist in four or more domains, so dedup-by-title is a loaded weapon on a generic title.

Sources notes merge group by group rather than appending, so re-filing under an existing heading adds to it instead of writing a second copy beside it. Near-duplicate flags carry both notes' heading sets, so the finding can be judged without reading both bodies. A new section-edit-risk flag answers "will a targeted edit land where I meant", which goes wrong earlier than "can this be read whole" and is invisible in a size warning.

start() now serves preferences and protocols in full — the two a session needs whole to orient before it knows enough to ask for them — while the other three stay at heading depth. session() reports what is actually pending rather than reciting every step, and adds a cross-singleton audit: not just whether the five singletons agree, but whether the operating rules still serve what the goals describe. A protocol can be perfectly consistent and still be serving a goal that has moved.


176 tests, tsc --noEmit clean, build clean. 41 core tools (was 39), 74 with full mode.