v2.4.0

Hatchdoor v2.4.0

Important

Container deployments must update their environment and bind mounts before upgrading. The old MCP attachment-inbox ingestion system has been removed, a persistent models mount is now required, and the first v2.4.0 startup rebuilds and re-embeds the generated cache.

This is a feature-packed release that I hope you will enjoy! Tons of new things, especially if you are using the LLM Wiki pattern (see below). A bit of prep for you to do (now direct sending of attachments via the MCP) and a new reindex, but this should be the last time in a while, and it is for the best. New embeddings model for better and faster results, and smaller RAM footprint of the server (from 1.3GB to 500MB on reindex).

Enjoy!

Required upgrade actions

1. Replace the retired attachment-inbox mount

Remove the old attachment-ingestion configuration from .env and Docker Compose:

• HOST_ATTACHMENT_STAGING_PATH
• HATCHDOOR_MCP_ATTACHMENT_STAGING_PATH
• HATCHDOOR_MCP_ADVERTISE_HOST_PATHS
• The attachments-inbox bind mount

Agents must now upload files directly: use multipart POST /api/attachment by default, or the base64 import_attachment MCP tool when they cannot make an out-of-band HTTP request.

2. Add the persistent models mount

Add this to .env:

HOST_MODELS_PATH=./models

Ensure Docker Compose mounts it at /models:

${HOST_MODELS_PATH:-./models}:/models

This directory persists downloaded search models and the local EmbeddingGemma terms-acceptance receipt across container restarts and image upgrades.

Use the checked-in .env.example and docker-compose.yml as the canonical configuration examples.

3. Expect a full reindex on first startup

v2.4.0 rebuilds the disposable SQLite cache automatically. The cache schema changes from 7 to 8, FastEmbed changes from v4 to v5, and embeddings now include each note title and heading path. Hatchdoor therefore reindexes and re-embeds the vault after upgrade. Markdown source files are not changed.

The first startup may also require selecting and downloading a search model. Keep the new /models mount in place so this work is not repeated after a restart.

4. Review renamed and new environment variables

• Rename HATCHDOOR_MCP_MAX_ATTACHMENT_BYTES to HATCHDOOR_MAX_ATTACHMENT_BYTES for the HTTP/web upload limit.

• HATCHDOOR_MCP_MAX_BASE64_BYTES controls the decoded-size limit for base64 MCP uploads.

• HATCHDOOR_EXCLUDE is a comma-separated, gitignore-style list of paths to leave out of indexing, for example:

  HATCHDOOR_EXCLUDE=imports/,*.bak

• HATCHDOOR_EMBED_LAYERS=false leaves demoted layers keyword-searchable while skipping their vector embeddings.

Note

.hatchdoor-layer is not an exclusion file. It marks a folder as a named, demoted layer. Use HATCHDOOR_EXCLUDE to exclude folders or files from the index.

LLM Wiki raw-source layers

Andrej Karpathy’s LLM Wiki pattern separates immutable raw sources from the LLM-maintained wiki. Hatchdoor can keep those raw files on a separate, explicitly selectable surface:

# raw/.hatchdoor-layer
raw

The raw/ folder then stays out of the browser and default search results, while an MCP client can select it with layers: ["raw"]. Pair this with HATCHDOOR_EMBED_LAYERS=false when the raw corpus should remain available for keyword lookup without consuming vector-indexing resources.

A layer does not make files read-only; use your agent schema/instructions to preserve the raw-source immutability convention.

LLM Wiki users: follow the step-by-step raw-source layer setup before configuring your agent workflow.

Highlights

• Vault layers and diagnostics: keep source material on named, demoted surfaces; MCP clients can select them explicitly while browser routes remain on the default surface.
• Noise exclusions: built-in rules now omit .obsidian/, .trash/, .hatchdoor-trash/, .DS_Store, *.tmp, and *.sync-conflict-*; deployment-specific rules use HATCHDOOR_EXCLUDE.
• First-run semantic-search setup: guided EmbeddingGemma terms acceptance, download progress, and indexing, with Nomic Embed Text v1.5 as an English-focused fallback.
• Direct agent attachment uploads: no shared staging folder; use HTTP by default or base64 MCP upload as fallback.
• Inline PDF previews: linked PDF vault assets now render directly in the app.
• Improved retrieval: FastEmbed v5 and contextual embeddings include each note title and heading path.

See CHANGELOG.md and the README’s Vault Layers And Exclusions section for the complete configuration details.

v2.3.0

Important upgrade notice: cache rebuild on first start

v2.3.0 changes Hatchdoor's generated SQLite cache and embedding pipeline. A full cache rebuild is required and happens automatically on first start.

Why the rebuild is required

• Nomic Embed Text v1.5 now uses a 1,024-token input window. Earlier Hatchdoor versions did not set the model's maximum input length, so FastEmbed silently applied its generic 512-token default. This truncated longer chunks at 512 tokens during embedding.
• v2.3.0 explicitly configures the Nomic model for 1,024 tokens and includes the maximum token length in the persisted embedder identity. Since sequence length changes the vectors for chunks that exceed the old limit, existing vectors cannot be reused safely.
• The cache schema also advances to version 6 to store structured note-level frontmatter metadata.

On upgrade, Hatchdoor detects the schema and/or embedder-identity change, wipes the generated SQLite cache, and rebuilds it from the Markdown vault. This is expected. The UI remains reachable and shows live indexing progress, but vault data, search, assets, writes, and MCP endpoints stay unavailable until the rebuild commits successfully. Large vaults may take some time to finish; wait for the index to complete before using the API or MCP tools.

Startup and indexing

• The UI is available while the initial vault index is built, with token-weighted progress, note and chunk counts, elapsed time, and a measured ETA.
• Indexing now logs regular human-readable progress heartbeats and additional performance diagnostics.
• Embeddings are processed chunk-by-chunk rather than padded to the largest item in a batch, reducing peak memory pressure and avoiding wasted inference work.
• Vault watching and Git sync begin only after the initial index is ready.

MCP, metadata, and search

• YAML frontmatter is parsed into typed note metadata.
• get_note now returns normalized tags, aliases, and frontmatter properties.
• search_notes supports exact tag filters, hierarchical tag-prefix filters, vault-relative path prefixes, required property names, typed property equality, and explicit property projection.
• query_notes adds metadata-only note queries for requests that do not need content retrieval.
• Hierarchical tag matching lets topic/selfhosting match notes tagged topic/selfhosting and descendants such as topic/selfhosting/immich.
• UI tag searches now use #tag/path syntax and return the matching tag branch.

Frontend and iOS PWA

• Added a regression test preserving viewport-fit=cover, ensuring iOS safe-area insets remain available in the installed PWA on notched and Dynamic-Island devices.

Dependencies and quality

• Refreshed Rust and frontend dependencies, including git2 0.21, resolving the current RustSec and npm audit findings.
• Added indexing diagnostics and microbenchmark coverage, startup-gate coverage, metadata regression coverage, and hierarchical tag-search coverage.

Documentation and contribution workflow

• Added and consolidated architecture decision records.
• Refreshed contributing guidance for local setup, checks, security, licensing, and the development-branch PR workflow.

Full Changelog: v2.2.0...v2.3.0

v2.2.0

Hatchdoor v2.2.0 is a major release. It introduces write mode (full note editing from the web UI and through MCP tools), then hardens the backend after a complete robustness and client edge-case audit, adds a read-only demo mode, and ships a public-ready distribution.

✍️ Write Mode (new)

Hatchdoor is no longer read-only. You can now create and modify vault content from the web app and let agents do the same over MCP.

• Inline note editor with live preview, drafts, and a contained mobile editing experience.
• Note management dialogs: create, rename, move, archive, and delete notes.
• Conflict review: concurrent edits are detected and surfaced for review instead of silently clobbering.
• Write API: new backend note write endpoints backing the editor.
• MCP write tools: create, edit, append, replace-section, move, rename, archive, and trash notes, plus attachment import, all vault-safe and exposed to agents.
• Image uploads normalized to WebP (respecting the encoded image type).
• Markdown quality warnings reported on write.
• Renamed notes return their new slugs over MCP so callers stay in sync.

🔒 Security & Hardening

Following a full backend robustness audit:

• MCP now requires a bearer token whenever it is enabled, so there is no unauthenticated tool access.
• The server refuses to start unauthenticated on a public bind.
• Access tokens are redacted from request trace spans.

🛡️ Reliability & Data Safety

• Git sync: recovers from an interrupted merge before syncing, runs fetch/push outside the vault write-lock, commits the whole working tree each sync, and retries transient failures on a backoff.
• Cache: self-heals notes after a per-note embedding failure, recovers a poisoned writer mutex instead of wedging, rebuilds the vector index when the embedder model changes, and rebuilds a half-initialised schema instead of bricking startup.
• Vault writes: rename-first, all-or-nothing asset moves on delete; fsync of the temp file and parent dir in atomic writes; rollback of multi-file rewrites when one fails; safer trash-collision handling.
• MCP protocol: negotiates the version instead of exact-match lockout, and reports "note not found" as a proper tool error on write tools.
• HTTP: raises the attachment body limit to the configured max and preserves JSON rejection status codes.

🖥️ Frontend Fixes

• The service worker no longer auto-reloads mid-edit and destroys unsaved changes.
• The image normalizer now applies EXIF orientation instead of re-encoding blindly.
• The graph page no longer redraws at 60fps with per-frame O(n log n) work.
• Read failures now surface the server's error body.

✨ Other Features

• Read-only demo mode for safe public deployments.
• Starter docs seeded automatically for empty vaults.

🧱 Refactors & Internals

• Split oversized backend modules per the architecture audit.
• Extracted App hooks, regrouped frontend modules, and split the graph simulation.
• Fixed the sqlite_vec extern-fn transmute to use c_char instead of i8.

📦 Docs & Packaging

• README overhaul: screenshots, live demo link, table of contents, and agent-native / MCP-first framing.
• Docker: qualified base-image references, rootless/distroless image and Podman support documented, Docker Hub surfaced, and starter-vault seed docs bundled into the image.

---

Full changelog: v2.1.1...v2.2.0

v2.1.1 — MCP git sync, audit hardening, iOS PWA download fix

Merges development into main, covering everything since the last release on main (v2.0.0). This spans two version bumps — v2.1.0 (MCP git sync) and v2.1.1 (audit hardening + iOS PWA download fix) — 26 commits, 66 files (+6611/−509).

---

v2.1.1 — Audit hardening & iOS fix

Security, performance, and operational hardening from the 2026-06-11 codebase audit (audit-report.md), plus an iOS PWA download fix.

Fixed

• iOS PWA markdown downloads arrived as HTML. iOS ignores the <a download> attribute in standalone mode and treats the click as a navigation; Workbox's SPA navigation fallback then served the cached index.html for /api/note/<slug>/download. Added a navigateFallbackDenylist for /api/*, /vault-assets/*, and /health so those navigations reach the network. (Verified fixed on-device.)

Security

• Optional web API authentication via HATCHDOOR_WEB_BEARER_TOKEN: all /api/*, /vault-assets/*, and downloads require the token (Authorization: Bearer header or access_token query param); the PWA prompts on 401. (F-01)
• Default bind host changed to 127.0.0.1; 0.0.0.0 is now explicit opt-in. (F-01)
• Coalesced concurrent /api/refresh requests to prevent overlapping full reindexes. (F-02)
• Constant-time comparison for MCP and web bearer tokens. (F-06)
• SVG vault assets served with Content-Security-Policy: sandbox + Content-Disposition: attachment. (F-09)
• Stopped leaking absolute paths / raw internal errors in HTTP error bodies. (F-10)
• Capped POST /api/resolve-batch at 200 targets. (F-11)

Performance

• Moved reindexing, embedding, and query embedding off the async runtime via spawn_blocking, holding the cache write lock only for the final swap. (F-03)
• MCP write tools resolve the target note from the SQLite cache instead of rebuilding the full vault index from disk per write. (F-04)
• Enabled SQLite WAL mode + a pooled set of read connections for parallel reads. (F-05)

Correctness & operations

• Validate McpConfig once at startup (fail fast when write mode lacks a bearer token). (F-07)
• Git sync refuses to force-checkout over uncommitted manual edits to tracked vault files. (F-08)
• Hard-coded 90-archive/ prefix moved to HATCHDOOR_ARCHIVE_PREFIX. (F-12)
• Added a Forgejo Actions CI workflow (backend fmt/clippy/test; frontend lint/typecheck/test/build) and a Compose healthcheck. (F-13)
• SSE vault-events stream emits current revision on broadcast lag so slow clients resync. (F-16)
• /health runs SELECT 1 against the cache; binary gained --healthcheck mode for the container probe. (F-17)

---

v2.1.0 — MCP git sync

Added

• Optional automatic git sync of the vault: successful MCP write tools commit and push to the configured remote with debounced batching, conflict-abort semantics, and an immediate startup flush of stranded commits.
• get_git_sync_status MCP tool and a git-sync warning on write-tool responses.
• edit_note and replace_section MCP write tools.

Changed

• Richer git sync status reporting with plural-aware commit messages.
• Pinned the Rust toolchain to 1.96.0 and reformatted the tree.

Fixed

• Enabled the git2 https feature for TLS remote transport.
• Vault watcher ignores .git/ so sync churn does not trigger reindexing.

---

Test plan

• cargo test (backend) and npm test (frontend) pass — enforced by the new CI workflow
• Built v2.1.1 image and deployed to batterbrain; /health returns 200
• iOS PWA markdown download now returns the .md file (no longer HTML)

v2.0.0

What's new

MCP write tools

• Full vault write surface: create, update, append, move, rename, and delete notes
• Staged attachment tools: import, move, rename, and delete attachments
• Hardened mutations with safety checks (reject empty content, empty-path guards)

SQLite cache layer

• Persistent SQLite cache replaces in-memory snapshot state
• Incremental refresh — only re-indexes changed notes
• Schema versioning with migrations (v1 → v5)
• FTS5 full-text search on notes and chunks
• sqlite-vec extension for chunk vector storage

Hybrid semantic search (Phase 2)

• Nomic v1.5 embedder (768-dim) replaces BGE Small
• Hybrid retrieval: BM25 keyword + semantic vector search with per-note result cap
• Context assembly orchestrator with chunk-level scoring
• Redesigned search UI: note-grouped results with markdown stripping, scroll-to-matched section on open
• MCP search_notes and GET /api/search wired to the Phase 2 pipeline

Evaluation harness (Phase 1.5)

• eval binary with build, run, rerank, and compare subcommands
• Metrics: precision, recall, rank pre/post, latency stats
• Markdown report writer for repeatable benchmark tracking
• FastembedReranker (Jina v1 turbo + v2 multilingual) evaluated; hybrid retrieval selected over neural reranker

Graph page

• Force-directed knowledge graph rendered on canvas
• Zoom-adaptive label threshold — fewer labels when zoomed out
• Label deconfliction: labels never overlap nodes or each other at any zoom level
• Touch support, smooth zoom, and compact mobile header with slide-down tag filter overlay

Stats page

• Note counts, tag distribution, and last-modified list
• Ledger design; /api/stats and /api/graph backend endpoints

Brand refresh

• SVG wordmark in topbar replaces PNG logos
• Full favicon and PWA icon set regenerated from SVG source
• PWA manifest background_color and theme_color aligned to brand tokens

Theme toggle

• Light / dark / system theme toggle in topbar
• Mermaid diagrams render with dark theme in dark mode

---

Fixes & polish

• Mobile: search card text overflow, hotbar safe-area inset, drawer anchor to topbar, touch hover guards
• Frontend: callout redesign as pull-quote, collapsible toggle fix, task-list item renderer, h2 size reduction
• Vault watcher: live refresh on file events, handle event gaps and access events
• Docker: correct FASTEMBED_CACHE_DIR, add g++, pkg-config, and libssl-dev to build stages
• Wikilinks: heading and block anchor resolution; archived links marked visually in the graph
• Cache: inline tags must be namespaced (e.g. area/health)

---

Breaking changes

• SQLite vector dimension changed from 384 → 768. Delete the cache volume and let it rebuild on first start.
• In-memory app cache removed; SQLite is now the only runtime cache.

---

Previous release

See v1.1.0 for the prior changelog.

v1.1.1

v1.1.1

Patch release focused on stability, test coverage, and build reliability.

Fixes

• Stabilized searched note rendering when markdown rerenders while highlights are active.
• Moved search highlighting into the markdown rendering pipeline for more reliable hit navigation.
• Gated vault path helper re-exports to tests to remove a runtime warning.

Build

• Updated Docker builder images to current slim Trixie-based Rust and Node images.

Tests

• Expanded backend handler, cache, router, and SPA integration coverage.
• Added frontend regression coverage for navigation, search, links, downloads, storage, and note rendering.
• Enabled frontend coverage thresholds in Vitest.

Internal

• Split large backend, vault, frontend, note page, style, and test files into focused modules without intended behavior changes.
• Updated repository agent protocol documentation.

v1.1.0

Changelog

v1.1.0 - 2026-02-20

Compared with v1.0.0.

Added

• Added a Download .md action in the note actions menu.
• Added a server download endpoint: GET /api/note/{slug}/download.

Changed

• Switched markdown download flow to server-driven delivery for native mobile handoff.
• Updated frontend download trigger to use an anchor download flow instead of popup navigation.
• Added UTF-8-aware filename handling in Content-Disposition for markdown downloads.

Fixed

• Improved iOS/Safari file handoff behavior for .md downloads by using attachment headers.
• Added and updated frontend/backend tests for the download path and response headers.

v1.0.0

Hatchdoor v1.0.0

Hatchdoor v1.0.0 is the first release: a self-hosted, web-first Obsidian vault reader built with a Rust backend and responsive PWA frontend.

What’s Included

Core platform

• Rust server that serves both API and frontend directly
• Read-only vault browsing over web, optimized for desktop and mobile/PWA
• Obsidian wikilink support with robust note resolution

Vault navigation and discovery

• Collapsible vault tree explorer
• Recent notes
• Back/forward history
• Global vault search
• No mandatory “home note” model

Reader features

• Obsidian-style markdown rendering:
  • headings, lists, numbering, code blocks
  • callouts
  • tables
  • Mermaid diagrams
  • embedded images
• Frontmatter properties panel with tags
• Tag-to-search flow
• “On this page” table of contents
• Links panel (backlinks/outgoing links), collapsible and collapsed by default
• In-note search with match navigation

UX/UI

• Full Notion-inspired visual refresh
• Improved sidebar scanability and hierarchy
• Better mobile top bar and actions
• Improved iconography for notes/folders and larger folder disclosure controls
• Responsive behavior tuned for iPhone/PWA usage

Security and reliability

• Hardened path resolution and vault asset access controls
• Better handling of broken links and indexing edge cases
• Reduced redundant frontend polling updates when content is unchanged

Ops and developer experience

• Structured logging/tracing added
• Dockerized deployment with Cargo Chef multi-stage build and distroless runtime
• Compose/env-driven runtime configuration
• Expanded regression tests across backend and frontend

Notable fixes included before 1.0.0

• Mobile overflow/layout regressions in notes and code/table content
• Table semantics and autosize issues on mobile
• Code block copy reliability across browsers/contexts
• External links opening in a new tab safely
• Favicon/PWA icon and branding polish

Quality gates used

• Rust: cargo fmt --check, cargo check, cargo test, cargo clippy --all-targets --all-features -- -D warnings
• Frontend: format check, lint, typecheck, tests, production build

---