v1.5.6

Changelog

All notable changes to The Librarium are documented here.

[1.5.6] — Brains/documents split, document commands, research, knowledge base

This release covers four rounds of work: separating Librarium's internal data from user-facing documents and fixing document-writing detection; adding multi-site research; adding a persistent, incrementally-updated knowledge base; and upgrading research into a folder-aware tasking system with a documents tidy-up command.

Added

Brains / documents separation

• New librarium/documents/ folder — everything created or edited via a document command lives here, organized into topic subfolders.
• All internal state moved under librarium/brains/: librarium/brains/memories/, librarium/brains/temp-memory/, librarium/brains/notes/, librarium/brains/skills/.
• One-time, best-effort migration (migrateToBrainsLayout in main.ts) moves an existing install from the old flat librarium/* layout to the new split, guarded so it only touches installs still on the original default paths.
• New documentsFolder setting (default librarium/documents).

Document commands (src/fileCommands.ts)

• Replaced the old strict, English-only, adjacency-based regex gate (FILE_COMMAND_HINT) — which silently swallowed paraphrases like "make this a document", "save this", "put this in my documents", and anything not in English — with a broad, intentionally over-inclusive multilingual recall net (English, Spanish, French, German, Portuguese, Italian, Russian, Hindi, Chinese, Japanese, Korean, Arabic). The actual yes/no decision is now always made by the LLM classifier behind it, not the regex.
• Added an explicit field to the detected intent: explicit, non-destructive requests ("create a document for this") are now applied immediately and reported in chat; ambiguous requests, and anything destructive (delete/move), are still staged behind a confirm card.
• Before creating a new document, a semantic check (findMatchingDocument) looks for an existing document under documentsFolder that's clearly the right place for the content, to avoid near-duplicates.
• New chat-visible operation logging: an onNote callback posts important steps ("Searching existing documents…", "Found X. Updating it.", "✓ Created X.") as their own Librarium: ... chat messages instead of only flashing through the transient status line.
• Document commands can never see or propose changes to Librarium's own librarium/brains/ data.

Research (src/researchCommands.ts, src/researchPlan.ts)

• New explicit research feature: natural language ("research X and write up what you find") or a direct /research <topic> / /research <url> ... topic> command.
• Searches more sites than a skill lookup (researchMaxSites, default 12), groups fetched pages by topic (groupPagesByTopic) to detect when a search term covers genuinely different things (e.g. "Apple" the company vs. the fruit), and asks the user to disambiguate via a checklist card ("Write it up" / "All of them" / "Cancel") before writing anything, when needed.
• Structured/tasking upgrade: planTopicStructure decides, per resolved topic, whether it needs a single flat document or a folder of subtopic files (and, for broad topics, one further level of sub-subtopics) — e.g. "research and construct a pandas documentation folder" now produces documents/pandas/dataframes.md, series.md, io.md, etc., each individually researched with its own targeted search, plus a generated overview file linking them. Depth/breadth is decided by the model, bounded by two hard caps: researchMaxSubtopicDepth (default 2) and researchMaxSubtopicFiles (default 12).
• New settings: enableResearch, researchMaxSites, researchMaxSubtopicDepth, researchMaxSubtopicFiles.

Knowledge base (src/knowledgeTypes.ts, src/knowledgeIndex.ts, src/knowledgeExtraction.ts, src/knowledgeBase.ts)

• New persistent, incrementally-accumulated knowledge layer underneath memory topics: entities, atomic facts, typed relationships, and attributes, each fact/relationship carrying confidence, timestamps, and evidence (source + short quote) pointing back to the original document.
• Entity resolution is bounded, never a full KB scan: a token inverted index (KnowledgeIndex, over names/aliases/types/attributes/relationship-types) narrows candidates first; exact name+type match merges immediately; otherwise embedding similarity ranks the shortlist, with a bounded LLM disambiguation call only for borderline cases. This is what keeps e.g. "Apple Inc." (technology company) and "apple" (fruit) as distinct entities.
• Conflicting facts are kept side-by-side (supersededBy), never silently overwritten.
• Only the entities a document actually touches get their compact profile (entity.summary) regenerated — an unrelated entity elsewhere in the KB is never re-derived.
• Query is a bounded hybrid lookup: index shortlist → embedding-ranked top entities (knowledgeMaxEntitiesPerQuery, default 5) → their own facts/relationships/evidence by direct id lookup — never a scan of the whole KB, so query cost stays roughly flat as the KB grows.
• New settings: enableKnowledgeBase, knowledgeMaxEntitiesPerQuery.
• Manual-only updates: the knowledge base is never touched automatically by writing a document. It updates only via an explicit "Knowledge base: rebuild from documents folder" command-palette action or a /rebuild-knowledge chat command, both of which reread every document currently under documentsFolder. A per-document content hash (sourceHashes) means a document whose text hasn't changed since the last rebuild is skipped, so repeated rebuilds only pay for what's new or edited.

Documents tidy-up (src/documentsTidy.ts)

• New command — natural language ("tidy up my documents", "organize my documents folder") or a direct /tidy-documents command — that rereads every document under documentsFolder and proposes a folder/subfolder reorganization by topic, reusing existing subfolders where they fit and leaving anything that doesn't clearly belong to a group untouched.
• Expressed as ordinary move actions, so it reuses the existing file-action confirm card with no new UI — nothing moves until "Do it" is tapped.

Changed

• Settings UI reorganized with new groups: Documents (user-facing files), Research, Knowledge base, reflecting the new architecture. Folder-setting descriptions updated to distinguish "brains" (internal) from "documents" (user-facing) throughout.
• Orchestrator constructor now also takes a KnowledgeBase instance; main.ts constructs and persists it alongside the other stores.

Fixed

• The root cause of "write this into a file" sometimes returns the content in chat instead of creating a file: the old keyword-adjacency regex gate rejected the request before the LLM classifier ever ran. Detection is now recall-oriented at the regex stage and semantic at the LLM stage (see "Document commands" above).

---

Earlier history

Prior to this release, see README.md for the plugin's original feature set: topic-based memory with progressive abstraction layers, hierarchical multi-layer routing, note-memory mirrors, chat-history digest, grep search, basic web crawling, and skills (reusable how-tos built from web research).

Full Changelog: 1.0.6...1.5.6