Skip to content

Releases

v1.5.0 — task listings follow every edit, opt-in conflict-copy exclusion, TEI API key

Choose a tag to compare

@maxkuminov maxkuminov released this 13 Nov 04:45
85e8df5

Three additions and a round of fixes. There is no migration, and no tool was added, removed or changed in schema or description: the 32-tool contract is the same as v1.4.0.

New

  • Task listings follow generic edits. list_tasks and task_agenda read a derived table. Until now only the task tools refreshed it, so a task changed through edit_note appeared in listings only after the next index pass, up to five minutes later. Now edit_note, set_frontmatter and write_file to a note path refresh the written note's task rows right after publishing, and delete_note clears them. Each tool's result text is unchanged. create_note, moves, uploads and imports still wait for the pass. (#432)
  • Sync-conflict copies can be left out of the index. INDEX_EXCLUDE_CONFLICT_FILES=true (default false) keeps copies such as Note.sync-conflict-20260101-120000-ABCDEFG.md or Note (conflicted copy 2026-01-01 120000).md out of search and listings. It matches six fixed patterns on the file name only: Syncthing, Nextcloud (current and legacy), Dropbox, Obsidian Sync, and the form reported in #392. The files stay in the vault and remain readable and writable by exact path; move_note to a normal name brings one back into the index. The dashboard shows how many are excluded. (#392)
  • TEI_API_KEY, for a Text Embeddings Inference server started with --api-key. It is sent as a bearer token to TEI_URL only and is never logged or shown. A 401 or 403 never stops the server: embedding is withheld and the log names the setting. (#426)

Fixed

  • A dry run of add_task, complete_task or update_task no longer follows "Dry run — nothing was written." with a past-tense line. It now says "Would add", "Would complete" or "Would update". Real-write output is unchanged.
  • A vault path that is not valid UTF-8 no longer makes a list or search tool fail after it had done its work. (#334)
  • Saving task settings from outside the panel with timezone, excluded_folders or excluded_note_tags absent no longer clears the stored value. The panel always sends all three, so its behaviour is unchanged.
  • Consent page, light theme: the title of the selected access level was white on a pale card and could not be read. (#445)
  • The search-evaluation recipe no longer files its dump among production backups, checks the restore, and exports from the restored copy. (#408)

Upgrading

  • No migration and nothing to re-index. No setting needs to change.
  • FastAPI 0.141.1 → 0.162.0. A whole-body JSON null sent to POST /api/keys or PUT /api/keys/{id}/limit is still a 422, but the error detail's type changed from missing to model_attributes_type. The image gains opentelemetry-api, which FastAPI now requires; with no OpenTelemetry SDK installed it does nothing.
  • pgvector below 0.8.4 now logs one WARNING at startup, because 0.8.3 and 0.8.4 fixed HNSW vacuum errors. The server still starts; the remedy is in DEPLOYMENT.md.
  • TEI_API_KEY: surrounding whitespace is stripped, and a value that still contains anything other than visible ASCII refuses startup. Over plain http to a non-loopback host (only possible with EMBEDDING_ALLOW_PLAINTEXT) the key is sent, with one WARNING.
  • Turning on INDEX_EXCLUDE_CONFLICT_FILES removes each matching note's index row, vectors, links and task rows on the next pass, and links to it become dangling. A note whose title exactly matches a pattern is excluded too. DEPLOYMENT.md lists the patterns.

Known limits

  • A task refresh that cannot run (the note's row stays locked for more than 2 seconds, or a newer write superseded it) is silent. The rows converge at the next index pass, as they did before this release.
  • With conflict exclusion on: write_file, import_from_url and uploads to an excluded name carry no notice (only create_note and move_note do), list_tasks(scope="all") does not list tasks in excluded copies, and the setting is deployment-wide, not per user.
  • TEI_API_KEY was not exercised against a live TEI server. Its documented behaviour follows the upstream router source at v1.8.3.
  • move_note prints paths unescaped, so a file name containing a line break can forge a result line (#442).
  • The limits listed for v1.3.0 still apply (#429 recurrence shapes, #430 moving a task, #431 the publish window).

Full changes: #437, #438, #440, #441, #443, #444, #446

v1.4.0 — task rule: say which checkboxes are tasks

Choose a tag to compare

@maxkuminov maxkuminov released this 11 Nov 03:08
e769fd3

A checkbox is not always a task. Checklists, runbooks, acceptance criteria and the step lists inside work tickets use checkboxes too, and a vault that AI agents write into fills up with them. Since v1.3.0 the task tools listed every unchecked box as an open task; in one real vault that meant 1,868 "tasks" of which 314 were tasks. This release lets each user say which checkboxes count.

New

  • A task rule, per user, on the control panel's Tasks card ("Which checkboxes are tasks"), with a server-wide default:

    Rule A checkbox is a task when
    all (default) always. Nothing changes for an existing deployment.
    fields it carries a task field (a due, scheduled, start, created, done or cancelled date, a priority, a recurrence rule) or one of your task tags.
    filter its text contains your global filter text, such as #task (the Obsidian Tasks plugin's own "global filter").
  • The rule applies to the default scope: list_tasks, task_agenda and add_task's duplicate and similar-task check. scope="all" still lists every checkbox, and the write tools can still act on any checkbox by id.

  • Nothing is hidden silently. Under fields or filter the response carries a line naming the rule and counting what it left out, for example Rule — task rule: fields — task tags: 1 [waiting] — excluded by rule: 2 tasks — … — pass scope="all" to list every checkbox. Rows the rule cannot decide for certain are kept and counted as unsure.

  • Under filter, add_task adds the filter text to the new line and says so. Under fields, a new line with no field or task tag is still written, and the response says default listings will not show it.

  • New settings: TASKS_RULE (default all), TASKS_RULE_TAGS, TASKS_GLOBAL_FILTER.

If agents write into your vault, choose fields or filter and say in your vault guide how a task is written, so agents create tasks the rule recognises. The README section "Which checkboxes are tasks" explains the choice.

Upgrading

  • Migration 040 adds three columns and two checks to task_settings. No data is written.
  • Nothing changes until a rule is chosen. Under all, responses and cursors are byte-identical to v1.3.0.
  • New startup refusals. An invalid TASKS_RULE, TASKS_RULE_TAGS or TASKS_GLOBAL_FILTER, or TASKS_RULE=filter without a filter text, stops the server and the migration step. In a .env file, quote a filter that starts with # (TASKS_GLOBAL_FILTER="#task"); unquoted after a space it is read as a comment.
  • A list_tasks cursor is refused once after a user's rule changes: list again without it.
  • While a rule is in force, task_agenda needs a response cap of at least 3,121 characters (2,800 otherwise; the default cap is 40,000).
  • Rollback: a restarted v1.3.0 image cannot start against migration 040. Roll forward, or run alembic downgrade 039 with the new image first, which discards saved rules.

See "Upgrading across migration 040" in DEPLOYMENT.md.

Known limits

  • The filter match is a case-sensitive substring, as in the Tasks plugin: #task also matches #tasks and an occurrence inside inline code. The plugin's "remove global filter from description" display option is not applied.
  • Under filter, the duplicate check recognises an existing task with the filter in front or as a trailing tag, not in the middle of its text.
  • The limits listed for v1.3.0 still apply (#429 recurrence shapes, #430 moving a task, #431 the publish window, #432 listings after a generic edit).

Full change: #435

v1.3.0 — task tools: richer task queries, an agenda, and one-line task writes

Choose a tag to compare

@maxkuminov maxkuminov released this 10 Nov 22:20
58564cd

Agents can now manage checkbox tasks without reading and string-editing notes: find them with real filters, see what is due in one call, and add, update or complete a single task line safely. It works on plain - [ ] text checkboxes; the Obsidian Tasks plugin's emoji fields are optional and nothing changes for a vault that does not use tasks.

New

  • list_tasks, extended. Date windows with a lower bound (due_from, scheduled_from, done_from / done_by), relative dates (today, tomorrow, +7d) in a configurable timezone, undated, has_due, text, priority, recurring, exclude_tags, path, order_by, a compact row (detail="brief"), the lines under a task (context), and a default scope. Every row now carries a task id.
  • task_agenda (read-only). Overdue, due today, scheduled and now due, and upcoming, each counted, from one snapshot, with an exact list_tasks call to continue any section.
  • complete_task, update_task, add_task (write). Each changes one task line, addressed by note path plus task id:
    • The id is derived from the task's own text. Nothing is written into a note to identify a task.
    • The id is resolved against the note's current bytes inside the call. A task that changed, moved or is ambiguous is refused with nothing written; identical lines in one note need the note's expected_hash.
    • Only the fields a call names change. Indentation, list marker, field order, tag positions and line ending are left as they were.
    • Completing a recurring task writes its next occurrence with the dates the Tasks plugin would compute. Supported rules: every day / N days, every weekday, every week / N weeks, every week on named weekdays, every month / N months, every month on the Nth or the last day, every year / N years, a named month on the Nth; each may end in when done. Any other rule is refused (recurrence_unsupported) and nothing is written.
    • add_task refuses an exact open duplicate and lists similar open tasks without blocking.
    • dry_run on all three; a listing reflects a task write immediately, without waiting for the index pass.
  • Per-user task settings on a "Tasks" card in the control panel (the Settings page in single-user mode): timezone, folders and note tags left out of listings by default, and whether to write done, cancelled and created dates and where a recurring task's next occurrence goes. Server-wide defaults: TASKS_DEFAULT_TIMEZONE, TASKS_SET_DONE_DATE, TASKS_SET_CANCELLED_DATE, TASKS_SET_CREATED_DATE, TASKS_NEXT_RECURRENCE_BELOW.

The server now publishes 32 tools; a read-only OAuth grant lists 20.

Upgrading

  • Migration 039 runs on upgrade: new nullable columns and a task_settings table. No data is rewritten and it runs as the non-superuser database owner.
  • One re-derive pass. The first index pass after the upgrade re-reads every note once to give each task its id. Nothing is re-embedded. Until it finishes, rows show id pending, the text and recurring filters are approximate and say so, and move_note(rewrite_links=True) answers index_not_ready.
  • New startup refusal. An invalid TASKS_DEFAULT_TIMEZONE stops the server and the migration step. Unset means UTC. Set the timezone that should define "today", either here or per user on the Tasks card.
  • What an existing list_tasks caller sees. Rows gain an id and the header gains task ids pending. tags= now matches nested tags and ignores case (work also matches #Work/admin). A tags value that can never be a tag (["2024"], ["a.b"]) is refused instead of returning an empty page. A cursor issued before the upgrade is refused once: list again without it. A path containing a backtick is shown with it escaped.
  • Connectors pick up the four new tools when they next list tools; an already-open session keeps its old list.
  • CLASS_MAPPING_VERSION moves to 6, so concurrency-readiness evidence gathered before the upgrade does not carry over. The image gains a pinned tzdata package.
  • Rollback is roll-forward. Downgrading migration 039 drops saved task settings.

See "Upgrading across migration 039" in DEPLOYMENT.md.

Known limits

  • Recurrence rules outside the list above are refused, including every 2 weeks on Monday, every month on the last Friday, every weekend, until … and for N times (#429).
  • There is no tool to move a task to another note (#430).
  • A task changed through edit_note or another generic write tool appears in listings after the next index pass, not immediately (#432).
  • The task write tools share edit_note's publish path: an edit made outside the server in the instant between its last check and its rename is overwritten (#431).
  • The Dataview inline-field task format and Tasks-plugin query blocks are not read.

Full change: #433

v1.2.0 — native Hugging Face TEI embedding provider

Choose a tag to compare

@maxkuminov maxkuminov released this 08 Nov 17:03
16e4a52

Adds an opt-in embedding provider for Hugging Face Text Embeddings Inference's native /embed API. Thanks to @jaimemachado, who contributed the provider in #424.

New

  • EMBEDDING_PROVIDER=tei, with TEI_URL, TEI_EMBEDDING_MODEL and TEI_EMBED_BATCH_SIZE. See the README for setup.
  • A startup check against TEI's GET /info. A proven mismatch of TEI_EMBEDDING_MODEL or batch size refuses to start. If /info cannot be reached, the server keeps serving and sends no embeddings until a check passes.

Upgrading

No action is needed unless you switch to tei. Ollama and OpenAI behaviour is unchanged. As with any provider change, switching an existing index needs make reset-embeddings.

Known limits

  • Input over the model's limit is truncated, as with Ollama.
  • A TEI server started with --api-key is not yet supported (#426).
  • Tested against TEI 1.8.3.

Full change: #427

v1.1.1 — ChatGPT connects via CIMD private_key_jwt

Choose a tag to compare

@maxkuminov maxkuminov released this 07 Nov 22:42
35815e4

ChatGPT can connect again. Its connector identifies itself with a Client ID Metadata Document that declares private_key_jwt, and until now that document was refused with invalid_client right after login. This release has two changes since v1.1.0:

  • private_key_jwt client authentication for CIMD clients (#420, #421): ChatGPT now authenticates at /token and /revoke with a signed client assertion.
    • The server fetches and verifies the client's public key set (jwks_uri, same origin as the document), stores it with its origin fetch time, and refetches it only when an assertion names an unknown key or the stored copy is more than 24 h old.
    • A failed refetch of a stale set falls back to the stored keys, so an outage at the client's origin does not break refreshes.
    • Assertions are single-use, have a 300 s lifetime and get zero exp leeway. jku/jwk/x5*/crit are refused, and RS256, PS256 and ES256 are supported.
    • CIMD only: /register still refuses private_key_jwt.
  • Docs (#423): the upgrade step for existing ChatGPT connectors (below), and the release-day deploy record.

Nothing changes for existing clients. Claude, Claude Code and other public CIMD or DCR clients, and their grants, are untouched, and a failed assertion never revokes anything. The MCP tool contract is unchanged: the 28 tools, their schemas, result shapes and MCP-REFUSAL codes are byte-identical to 1.0.

Upgrade notes

  • One migration, 038. On Kubernetes it runs in the init container; on compose, make deploy runs it.
    • It adds four nullable columns to oauth_clients (jwks_uri, token_endpoint_auth_signing_alg, jwks_snapshot, jwks_fetched_at), replaces two CHECKs and adds one. There is no backfill, and it takes seconds.
    • The previous image runs against the 038 schema.
    • Its downgrade refuses while any private_key_jwt client exists. Delete that client on the panel's OAuth page first; its grants go with it.
  • No new settings.
  • The AS metadata gains private_key_jwt in token_endpoint_auth_methods_supported and adds token_endpoint_auth_signing_alg_values_supported (RS256, PS256, ES256), only while OAUTH_CIMD_ENABLED is true. With it false, the metadata is byte-identical to 1.1.0.
  • Re-add an existing ChatGPT connector; don't just reconnect it.
    • ChatGPT reads the server's OAuth metadata once, when a connector is created, and picks its token authentication method from that copy.
    • A connector created before this release authenticates as a public client and is refused at /token (oauth_token_refused reason=invalid_client.assertion_missing).
    • Delete it in ChatGPT and add it again with the same URL. Nothing changes on the server.
    • This happened on the reference deployment on release day.
  • Outbound access. Besides each client's document, the server now fetches a private_key_jwt client's jwks_uri at consent and, when needed, before a token request. It uses the same SSRF-guarded client and egress rules as the document fetch. The refetch also runs while OAUTH_CIMD_ENABLED=false, because it only fetches a URL pinned on an approved client.
  • Verifying a ChatGPT connection: look for oauth_consent_granted registration_kind=cimd cimd_host=chatgpt.com, then oauth_token_issued. The full check is in DEPLOYMENT.md, and the rules and accepted limitations are in docs/architecture/oauth-and-grants.md.

Known follow-up

OpenAI recommends that authorization servers accept any token authentication method in the intersection of the client's token_endpoint_auth_methods_supported and their own. This release deliberately honours only the declared singular method, so a private_key_jwt client can never fall back to an unauthenticated public flow. If OpenAI drops that transitional singular field, ChatGPT may choose none. That decision is tracked in #422.

v1.1.0 — native OIDC login with admin-managed account linking, Keycloak reference

Choose a tag to compare

@maxkuminov maxkuminov released this 05 Nov 15:41
7e8c992

The control panel can now sign people in through an OpenID Connect provider, with Keycloak as the reference. This release has four changes since v1.0.0:

  • Native OIDC login for existing accounts (#415): the panel signs people in through any standards-compliant OpenID Connect provider.
  • Admin-managed account linking (#417): an admin links each local account to a provider identity, and the first matching sign-in pins it.
  • A docs fix for an activation trap (#418): the OIDC client secret has to reach every process that loads the settings.
  • A cleaner login page when SSO is on (#419).

Nothing changes unless you set OIDC_ENABLED=true. Single-user deployments, and multi-user deployments that sign in with passwords, behave exactly as before. The MCP tool contract is unchanged: the 28 tools, their schemas, result shapes and MCP-REFUSAL codes are byte-identical to 1.0.

Upgrade notes

  • One migration, 037. On Kubernetes it runs in the init container; on compose, make deploy runs it. It adds six nullable oidc_* link columns and five constraints to users, with no backfill, and takes seconds. It does not refuse startup and needs no rebuild or re-embed. Its downgrade drops every link.

  • New settings. The defaults need no action:

    • OIDC_ENABLED (false)
    • OIDC_ISSUER
    • OIDC_CLIENT_ID
    • OIDC_CLIENT_SECRET
    • OIDC_TOKEN_AUTH_METHOD (client_secret_basic, or client_secret_post)
    • OIDC_REQUIRED_GROUP (optional)
    • OIDC_EXTRA_SCOPES
    • OIDC_DISPLAY_NAME (SSO)
    • OIDC_CA_FILE
    • LOCAL_LOGIN_ENABLED (true)

    All of them are validated at startup. That validation never contacts the provider, so a provider outage cannot stop the server from starting. They are documented in the README Configuration table, in .env.example, in DEPLOYMENT.md and in docs/native-oidc.md.

  • Before you set OIDC_ENABLED=true, give the secret to every process that loads the settings. That means the migration step (alembic upgrade head, or the Kubernetes migrate init container), alembic check, and one-off Jobs, not just the server. Each of them refuses to start without OIDC_CLIENT_SECRET, with OIDC requires OIDC_CLIENT_ID and OIDC_CLIENT_SECRET.

    • The example manifests load one Secret with envFrom everywhere, so they are already safe.
    • If you inject the secret into the app container only (env with a secretKeyRef), the first rollout crash-loops in the migration step. Under Recreate the old pod is already gone, so the server is down.
    • This happened on the reference deployment on activation day. OIDC_ENABLED=false restores service while you fix it.
  • Turning SSO on. There is no SSO bootstrap: while OIDC is on, /admin/register answers 404. Follow this order:

    1. Create the first admin with a password, before you turn SSO on.
    2. Register a confidential client at the provider with the exact redirect URI <BASE_URL>/admin/auth/oidc/callback.
    3. Set OIDC_ENABLED=true and keep LOCAL_LOGIN_ENABLED=true.
    4. Sign in with the password and link each account on its edit page, under Single sign-on: Expect email (a verified email) or Bind subject.
    5. Have each user sign in once with the SSO button.

    The Keycloak walkthrough covers the provider side.

  • An unlinked identity is refused. Nothing is provisioned just in time, and the first SSO sign-in never creates an account or an admin. A refusal looks the same whatever the cause. The reason is only in the security log (panel_oidc_refused, field reason).

  • Password login stays on by default. Set LOCAL_LOGIN_ENABLED=false only after every admin is linked and the startup log shows no CRITICAL lockout line. Turning it back on and restarting is the break-glass. While password login is off, the panel refuses any change that would leave no admin who can sign in.

  • Remove an external login gate only after SSO works end to end. This means forward auth or basic auth in front of the panel. Do not remove it by switching off MULTI_USER_MODE.

  • Access logs. Suppress query strings on /admin/auth/oidc/callback in your reverse proxy, ingress and tracing, because the query carries the authorization code.

  • If you ran a pre-release build of #415 with OIDC_SUBJECT_USER_IDS: that setting is retired, and any non-empty value refuses startup. Move the pairs into the panel with Bind subject (docs/native-oidc.md, "Upgrading from OIDC_SUBJECT_USER_IDS"). Upgrades from v1.0.0 never had it.

Changes

  • Native OIDC login (#415, closes #414). The app stays the MCP OAuth authorization server, and the provider only signs people in to the panel and the consent page. The flow is the authorization code flow with PKCE S256, state and nonce, and RFC 9207 iss checking. ID tokens are validated against the provider's JWKS (fetched per login), with zero clock leeway. Pending logins live in the single worker's memory, so --workers 1 stays part of the contract.
  • Account linking (#417, closes #416). Each local user can have an expected identity, either a verified email or a subject. The first matching sign-in permanently pins the account to (issuer, subject). An expectation expires after 7 days.
    • Admins manage links from the user edit page (expect, bind, unlink). Every change is a security event (panel_oidc_linked, panel_oidc_link_set, panel_oidc_link_removed).
    • The provider profile is wider: RS256, PS256 and ES256 ID tokens (as advertised by discovery), client_secret_post, extra scopes, a private CA for the provider, and a configurable button label.
    • OIDC_REQUIRED_GROUP is now optional.
    • SSO sign-ins stamp the user's last-login time.
  • Docs: the OIDC client secret reaches every process (#418). This covers native-oidc.md (Settings, Troubleshooting), DEPLOYMENT.md, the Kubernetes guide (Configuration, Troubleshooting) and .env.example.
  • Login page layout (#419). With SSO on, the "Sign in with …" control is a full-width button, followed by an "or use a password" divider. The password button becomes the secondary style. With OIDC off, pages are byte-identical to before.

Full changelog: v1.0.0...v1.1.0 (#415, #417, #418, #419)

v1.0.0 — tool-contract freeze, scoped tools/list, CIMD, tags in keyword search, chunk minimum, reranker documents, search confidence

Choose a tag to compare

@maxkuminov maxkuminov released this 01 Nov 16:50
da0f11a

Eight changes since v0.10.0, and the first release whose version number promises the tool contract: hybrid_search and semantic_search pages carry confidence: ok | low, shipped off (#389); chunks have a 200-character minimum and end on word boundaries, CHUNKING_VERSION 2 (#390); tags are in the keyword vector and matched_in gains tags, migration 035 (#391); the docs recommend Qwen3-Embedding for local embeddings and describe the reranker reference setup (#405); tools/list follows the credential, so a read-only OAuth grant lists 19 tools, and the 28-tool contract fixture is frozen (#358); the OAuth server accepts Client ID Metadata Documents with DCR as the fallback, migration 036 (#357); the search-evaluation harness gains a mechanical gate step (#411); and rerank=true sends each note's embedded text for up to four of its fetched chunks and scores the note on the best of them, with RERANK_MAX_DOCUMENTS (#402).

Upgrade notes

  • Two migrations, 035 and 036, run by the init container on Kubernetes or by make deploy on compose (which stops the app first). Neither refuses startup or needs make reset-embeddings. With Recreate, a direct upgrade from v0.10.0 is one downtime window.
  • Before upgrading, run the 035 preflight. DEPLOYMENT.md, "Upgrading across migration 035", has a read-only statement (counts and ids only; a one-config and a multi-config form). flagged must be 0. A flagged note is shortened (body or tags) and re-indexed by the current image first.
  • 035 (tags in the keyword vector) rewrites every tagged vector once, in SQL, under the index generation lock: about 30–40 seconds for 4,265 synthetic notes of 1,800 words with three tags each, added to the Recreate gap. It is all or nothing, with a size preflight that aborts naming row ids before any UPDATE. Under Recreate an abort is downtime until the image digest is reverted. If you migrate by hand, stop the app first: an old-image process that writes a vector after 035 commits writes it without tags. Nothing is re-embedded. make rebuild-tsvectors afterwards is optional (it makes shifted body positions byte-exact). Rollback: alembic downgrade 034 with the new image, then the old image.
  • 036 (CIMD) adds oauth_clients.client_id_metadata_url and the CHECK ck_oauth_clients_cimd_shape, with no backfill, in seconds. Its downgrade refuses while any CIMD client row exists: delete those clients on the panel's OAuth page first (their grants go with them, and their users reconnect).
  • Connectors switch to CIMD on their next fresh connection. The AS metadata advertises client_id_metadata_document_supported: true; claude.ai, Claude Code and ChatGPT then send an https client_id instead of calling /register. Existing DCR grants are untouched, and DCR stays the fallback for a client that sends no URL. Claude Code can now complete on a loopback callback.
    • The consent page names the application by its document URL's host, shows the full URL and "First approved", and does not render the document's self-declared client_name (panel-only). An all-loopback client gets an extra warning. Its CSP form-action gains http://localhost:* and http://127.0.0.1:*.
    • Every authorization redirect carries iss (RFC 9207). resource (RFC 8707) is classified in the security log and never refused; enforcement is #404, after 14 consecutive days with no resource_class: unexpected.
    • The server fetches each client's metadata document over https, through the import_from_url SSRF policy. An egress policy that blocks public HTTPS breaks CIMD (invalid_client; client_metadata.fetch_failed or client_metadata.ssrf_refused in the log).
    • Rollback lever: OAUTH_CIMD_ENABLED=false and a recreate, no rebuild. Clients fall back to DCR, and CIMD grants keep refreshing.
  • Read-only OAuth grants list 19 tools. A grant without readwrite no longer lists the nine write-required tools (create_note, edit_note, move_note, delete_note, set_frontmatter, write_file, delete_file, import_from_url, request_upload); calling one is still permission_denied. API keys of either permission and readwrite grants list all 28, byte-identical to before. No list_changed is sent: after a scope change in the panel, a connector sees the new list when it reconnects. For a read-only caller, the refusals write_file / delete_file made before their permission check (a malformed expected_hash, no_incumbent) are now permission_denied.
  • Chunking version 2 is a second gradual re-embed, with no migration and no setting. Every note is selected again under the existing per-tenant embed budgets; the index_transition: line returns on semantic_search, find_related and hybrid_search, and /health reads indexer.rechunk: "in_progress" until your notes drain. A section that did not change keeps its vector, and each pass logs Embedding complete: N notes, X chunks (S submitted, R reused). Startup does not refuse. Rollback: the previous image re-embeds every note stamped 2 in full.
  • New settings. The defaults need no action:
    • SEARCH_LOW_CONFIDENCE_SIMILARITY (0 = off, 0–1). Set it only after both tools' confidence tabulation lines PASS (DEPLOYMENT.md, "Calibrating search confidence"). The 0.50 candidate was measured on bge-m3 and is not calibrated for Qwen3.
    • RERANK_MAX_DOCUMENTS (100, 50–200; 50 restores the previous document count).
    • OAUTH_CIMD_ENABLED (true), OAUTH_CIMD_CACHE_MAX_SECONDS (86400, 0–86400), OAUTH_CIMD_FETCHES_PER_ADDRESS_PER_MINUTE (10, 1–120).
    • RERANK_TIMEOUT_MS now accepts 100–30000; the default stays 1,500.
  • Reranker. rerank=true now sends each candidate note's embedding input (title, heading path, tags, then the chunk, cut to 2,048 characters) for its cosine-best chunk and up to three more already-fetched chunks, at most RERANK_MAX_DOCUMENTS per call. A note scores the best of its documents, and a reranked row's preview and § locator follow the winning chunk. rerank=false is byte-identical. The usage row gains rerank_documents.
    • Rerank time grows with the characters sent (about 75 µs per character on the reference P4000, uncontended), so size RERANK_TIMEOUT_MS with the budget (DEPLOYMENT.md, "Reranker (optional)", has the sizing table). Measured on 2026-11-01 on that card with 2,000-character documents: 24 documents 3.2 s, 50 documents 6.7 s, 68 documents 9.1 s. A hybrid rerank=true call sends 67–68 documents on the reference vault, about 9–10 s uncontended, so the reference deployment runs RERANK_TIMEOUT_MS=20000 since 2026-11-01; the 10,000 ms it ran before timed out on every hybrid rerank call.
    • Rollback: set RERANK_TIMEOUT_MS to 10000 or less before redeploying v0.10.0, whose range ends there and refuses to start above it.
    • Under MCP_CONCURRENCY_MODE queue or enforce, keep RERANK_TIMEOUT_MS ≤ MCP_CONCURRENCY_WAIT_SECONDS × 1000 unless make concurrency-report passes with rerank=true traffic: a rerank=true call holds its embedding slot through the exchange.
    • The real-data gate (six reports, {semantic, hybrid} × depth 10/20/50, CI95 lower bound ≥ −0.01) passed four. Both failures are one cell, hybrid recall@20 class:identifier (mean +0.051, lower bound −0.0114, n = 66); every mean delta was ≥ 0. The owner waived it.
    • rerank=true against rerank=false, measured 2026-11-01 on the reference vault (196 mined queries, depth 50, the plain tool as baseline, paired CI95, no unreranked answer). On semantic_search it improves every top-of-list metric: overall recall@10 0.505 → 0.582 (+0.077 [+0.037, +0.116]), MRR 0.383 → 0.433, nDCG@10 0.389 → 0.442, with gains in both query classes. On hybrid_search it makes things worse: recall@20 0.791 → 0.731 (−0.060 [−0.110, −0.013]), MRR 0.533 → 0.465, nDCG@10 0.519 → 0.474. Plain hybrid is already the strongest ranking on that vault. The judgments come from what agents read after the existing tools' results, which may overstate hybrid's loss somewhat, but it does not turn into a gain. Keep rerank=false as the default; use rerank=true with semantic_search when the top few rows matter, and not with hybrid_search. Full table in DEPLOYMENT.md, "Reranker (optional)". Ranking quality is not part of the 1.0 contract; re-run the two reports on your own judgments after a model or reranker change.
  • Output changes clients may notice:
    • every hybrid_search and semantic_search page carries confidence: ok in its header (low, with a one-line hint, only once a threshold is set);
    • keyword_search finds a note by a tag alone, ranked below a title hit and above a body hit, and matched_in can name tags;
    • semantic previews start on a word, and a merged chunk's preview can include an inner heading line;
    • a read-only OAuth connector lists 19 tools.
  • Search evaluation. Reports are now schema_version 3; a report scored before this release must be re-scored before python -m scripts.search_eval gate accepts it. For a runfile: baseline, score the source generation before staging the runfile, and do not re-score it afterwards.
  • Embedding model guidance. The docs recommend Ollama qwen3-embedding:0.6b with a query instruction for local embeddings. The code default stays bge-m3, which remains supported; changing the model means make reset-embeddings and a full re-embed (README, "Embedding model choice").
  • MCP SDK stays 1.x (mcp[cli]==1.30.0); 2.x is internal and comes as 1.1 (#365).

Changes

  • Search confidence (#398, closes #389). confidence: ok | low and a hint on hybrid_search and semantic_search, from one pure rule (no keyword match on all terms and a top similarity below the threshold); no row is filtered or reordered. Usage rows record confidence, top_similarity and keyword_all whatever the threshold. The harness gains recall@20, recall@50 and a confidence tabulation.
    ...
Read more

v0.10.0 — list_tasks, change feed, hybrid_search, heading-aware chunks, opt-in reranker

Choose a tag to compare

@maxkuminov maxkuminov released this 31 Oct 04:43
425bc22

Nine changes since v0.9.3: list_tasks lists checkbox tasks from the index (#350); get_changes is a per-vault change feed, and get_recent/list_notes gain date ranges and cursors (#351); hybrid_search fuses keyword and semantic results (#354); semantic chunks follow headings and every vector-search row names its section (#355, #385); semantic_search and hybrid_search take an opt-in rerank (#367); scripts/search_eval/ measures search quality offline (#356); keyword_search prints rank legibly (#380); and two fixes, one for a transaction error under strict database TLS (#394) and one for the /health re-chunk state (#397).

Upgrade notes

  • Three migrations, 032 → 034, run by the init container on Kubernetes or by make deploy on compose (which stops the app first). With Recreate, a direct upgrade from v0.9.3 is one downtime window. None of them refuses startup or needs make reset-embeddings.

  • 032 (note_tasks) writes no rows. The extraction version moves to 4, so the first pass re-derives every note once and stores its tasks. Nothing is re-embedded. Until that pass finishes, list_tasks reports the unscanned count and move_note(rewrite_links=True) returns index_not_ready.

  • 033 (change feed) backfills notes_metadata.content_changed_at in SQL from modified_at. No re-derive, no re-embed. The feed starts at the first change after deploy, and nothing before 033 has events. An upgrade over an existing index writes no reset line. A reset appears only when a pass finds a scope with no indexed rows: a fresh install, a restore of a default (schema-only) backup, or a provenance discard.

  • 034 (heading-aware chunks) starts a gradual re-embed under the existing EMBED_*_BUDGET_* limits, in every mode. Old vectors keep serving until each note's new ones commit. Until your scope drains:

    • every semantic_search, find_related and hybrid_search response carries index_transition: N notes not yet re-chunked … after the header;
    • /health shows indexer.rechunk: "in_progress" (a state, never a count; status unchanged);
    • the dashboard's pending count falls each pass.

    On the reference deployment 4,206 notes across two vaults were pending at 20:44Z and 2,028 at 22:25Z: about half in 100 minutes, against a design estimate of 1.5–4 h for the whole drain. Rollback: alembic downgrade 033 with the new image, then deploy the old one.

  • Backups. note_tasks joins the derived tables whose data make db-backup and the Kubernetes CronJob leave out. An external job that copies the --exclude-table-data flags needs one for note_tasks too. The three feed tables are durable and always dumped. After a restore, a cursor taken later than the dump is refused.

  • New settings. The defaults need no action: CHANGE_FEED_RETENTION_DAYS (90), CHANGE_FEED_MAX_EVENTS_PER_SCOPE (100,000), CHANGE_FEED_MAX_EVENTS_PER_PASS (2,000), CHANGE_FEED_LOCK_TIMEOUT_MS (2,000), HYBRID_RRF_K (60), EMBEDDING_QUERY_INSTRUCTION (empty), RERANK_URL (unset), RERANK_MODEL, RERANK_TIMEOUT_MS (1,500), RERANK_ALLOW_PLAINTEXT (false). An out-of-range value refuses to start.

  • Tools 25 → 28 (list_tasks, get_changes, hybrid_search). The frozen tools_list_schema_5a391ca.json stays the prefix. New tools are in tools_list_schema_additions.json; the amended list_notes, get_recent and semantic_search schemas are in tools_list_schema_amendments.json. CLASS_MAPPING_VERSION moves to 5, so earlier make concurrency-report evidence does not carry over.

  • Output changes clients may notice:

    • keyword_search prints rank to three significant digits;
    • semantic_search and find_related rows are one line, with a § "Parent/Child" (#7) locator once a note is re-embedded, plus the transition line;
    • hybrid_search rows are JSON objects;
    • rerank is always accepted on semantic_search and hybrid_search; without RERANK_URL it returns reranked: false (not_configured);
    • move_note stops before rewriting links when its metadata transaction fails (db_failed warning).
  • Hotfix (#394). Under a strict DATABASE_SSL_MODE, the first session on a new pool connection could fail with ActiveSQLTransactionError: SET TRANSACTION ISOLATION LEVEL must be called before any query. It affected keyword_search since v0.9.3, and list_tasks, get_changes and hybrid_search. It was intermittent, mostly on parallel calls. The rollout is the only restart it needs.

  • MCP SDK stays 1.x (mcp[cli]==1.30.0; 2.x is #365).

keyword_search: rank with three significant digits (#381, closes #380)

  • rank renders with :.3g. A body-only cover scores about 1e-4, so pages used to read 0.000 beside a 2.660 fallback row.
  • The docstring now states the scales. Ordering and the forged-row protection are unchanged.

list_tasks (#382, closes #350)

  • - [ ], - [x] and - [-] lines (Tasks-plugin dates, priority, tags, heading path) are derived into note_tasks beside the link rebuild. Code is skipped, and query blocks are not evaluated.
  • Filters: folder, status (open excludes cancelled), inclusive due_by/scheduled_by, tags. The tool is owner-scoped in one snapshot, and its cursor carries no note id.
  • An extraction failure marks only that note (tasks_failed). It never quarantines the note. Limitations L1–L9 are in the archived design.

Search evaluation harness (#383, closes #356)

  • Judgments are mined from the usage log: a read_note within 120 s of a search by the same key or grant. Candidate arms are replayed against a restored BACKUP_FULL=1 copy and scored by recall@5/10, MRR and nDCG@10.
  • The export is tenant data. It goes to a git-ignored 0700 .search-eval/, and a multi-user report omits query text.
  • If a candidate arm lost corpus, the run exits CORPUS_LOSS (2) instead of reporting a score. CI runs a synthetic fixture.

Change feed (#384, closes #351)

  • get_changes(since, …) returns one line per created/modified/moved/deleted event, labelled with the agent's tool or human/sync. The ev:<seq> cursor is per vault.
  • Pruned or capped history is reported as a gap line or history_truncated, never silently dropped. Recording an event never changes a write's result.
  • get_recent and list_notes gain since, until, order_by and cursor. Limitations L1–L14 are in the archived design.

hybrid_search (#386, closes #354)

  • A third peer tool. It fuses keyword and semantic results by RRF over each arm's top 50, read in one snapshot. All three docstrings share one routing sentence.
  • The header shows arm_depth. A note outside both top-50s is not a candidate (L3).
  • On a provider failure the tool returns arms: keyword-only (semantic unavailable: <code>). The panel gains a hybrid_search analytics table.

Hotfix: strict-TLS probe left a transaction open (#394)

  • The pg_stat_ssl probe on each new connection opened a driver transaction that nothing ended. The listener now rolls it back before handing the connection out.
  • Regression tests cover a fresh connection and a growing pool.

Heading-aware chunks and section locators (#395, closes #355 and #385)

  • Chunks are cut at ATX headings with read_note(section=…)'s fence-aware grammar, and embedded with a title > H2 > H3 (+ tags) prefix.
  • Chunk count on the repository's own docs went 3,781 → 9,875; the design expects 1.3–2.5× on prose.
  • A per-note chunking version drives the re-embed, so startup never refuses. A title or tag edit now re-embeds the note (L4).
  • Queries are embedded with kind="query". EMBEDDING_QUERY_INSTRUCTION applies only to queries. Limitations L1–L11 are in the archived design.

/health: rechunk state before the first embed stage (#397)

  • Right after the 034 rollout, /health read complete with 4,206 of 4,274 notes pending, because a scope not yet counted rendered as zero pending. It now reads in_progress until every scope has been counted with zero pending.

Opt-in reranker (#396, closes #367)

  • With RERANK_URL set to a Jina-style /v1/rerank endpoint, rerank=true rescores at most 50 already-fetched candidates.
  • All or nothing: the page is either fully reranked or unreranked with a reason code. A reranker never fails a search.
  • A startup probe and background re-probe gate it, and /health reports rerank: enabled | disabled | absent. A CPU reranker times out at the default, so this release claims no quality gain.

Full changelog: v0.9.3...v0.10.0 (#381, #382, #383, #384, #386, #394, #395, #396, #397; registry and docs follow-ups #375–#379)

v0.9.3 — keyword_search finds titles, tool annotations, MCP Registry listing, lighter backups

Choose a tag to compare

@maxkuminov maxkuminov released this 29 Oct 10:41
88e6daa

Six changes since v0.9.2, all from the 2026-10-28 product and technology review (issues #343–#349): keyword_search finds notes by title and alias and returns snippets, every tool carries MCP titles and annotations, the server is listed in the official MCP Registry with a public image, database dumps stop carrying the rebuildable index, and the README leads with what sets this server apart in 2026.

Upgrade notes

  • Schema migration 031 (keyword_search, #344). It adds notes_metadata.body_excerpt and rewrites every keyword vector in place so title and alias lexemes carry weight A, under the index generation lock. 4,265 notes took a few seconds on the reference deployment. On the compose path make deploy now stops the application before a pending migration and starts it after; on Kubernetes the Recreate strategy already does this. Downgrade re-filters the vectors to body weight.
  • Run the keyword rebuild once after upgrading: make rebuild-tsvectors on compose, or deploy/kubernetes/base/rebuild-tsvectors-job.example.yaml on Kubernetes. Ranking is right immediately after the migration; snippets need the stored excerpt, which only a vector write stores. Until the rebuild runs, notes indexed before 031 show head: or no snippet, and the server logs the count at startup (N note(s) have a keyword vector but no stored excerpt). On a single-node cluster the Job can run with the app up; it serialises on the generation lock.
  • Three optional settings, defaults need no action: RANK_WEIGHTS (0.02,0.2,0.4,1.0; four finite numbers each in [1e-6, 1]), KEYWORD_EXCERPT_CHARS (20,000), KEYWORD_SNIPPET_ROWS (20). The snippet latency gate (300 ms) was measured at the defaults; cost scales with both.
  • Backups changed shape (#348). make db-backup and the Kubernetes CronJob now exclude the data of the five derived tables (notes_metadata, note_embeddings, note_links, indexer_runs, indexer_state): dumps are a fraction of the size and a restore is followed by one reindex from the vault. BACKUP_FULL=1 restores the old full dump; the pre-migration dump that make deploy takes is always full, because its point is an instant rollback.
  • Clients will prompt differently (#343). Eight tools are now declared destructive (edit_note, move_note, delete_note, set_frontmatter, write_file, delete_file, import_from_url, request_upload); create_note and request_download are additive; fifteen are read-only. Hosts that gate on destructiveHint will ask before those eight. Nothing about enforcement changed.
  • keyword_search output changed. The header now reads all terms: N, any term: M — without excerpt: K; each row carries match: all|any, matched_in: title|body|title+body, terms: x/y, and either snippet: with «» highlights or head: (the note's opening words). Bare terms are still ANDed; when the AND page is short, the server fills it with per-config OR matches, conjunctive rows always first. Quoted phrases, -term and explicit OR disable the fallback.

keyword_search: title and alias weighting, OR fallback, snippets (#374, closes #344)

  • Title and frontmatter aliases (also the legacy scalar alias, comma-split) are indexed at tsvector weight A, the body at D; ts_rank_cd with RANK_WEIGHTS ranks a title hit above any body hit. keyword_search("Decisions Log") now returns the note titled Decisions Log first; before this release it did not match at all.
  • Per-config OR fallback that is a superset of the AND page by construction; matched_terms counted per row.
  • Snippets via ts_headline over a stored 20,000-character excerpt, capped to the first KEYWORD_SNIPPET_ROWS rows; a title-only match renders head: rather than a fabricated highlight; snippet text is rendered on one line so a note cannot forge result rows.
  • The keyword transaction runs REPEATABLE READ, READ ONLY so the AND and fallback statements share one snapshot.
  • Both move paths (move_note and the indexer's rename detection) re-head the vector under the generation lock with a fingerprint re-read. The FTS config list is sorted everywhere so the migration and every writer produce byte-equal vectors.
  • Reviewed by an independent spec verifier and two adversarial Codex rounds; accepted limitations are recorded in openspec/changes/archive/2026-10-29-keyword-search-ranking/design.md (L13–L15).

MCP tool metadata (#373, closes #343)

  • Every tool has a title and the four ToolAnnotations hints; the server publishes instructions (what the server is, which tool to start with, the write-guard contract, the MCP-REFUSAL line).
  • Eight long descriptions restructured to stay under 4,000 characters, where Claude Code truncates; a snapshot test pins every description's length and the tools/list schema.
  • The write set is derived from the production module's __write_class__ stamps, not a hand-kept list.

Official MCP Registry and public image (#371, closes #346 and #178)

  • server.json (schema 2025-12-11, io.github.maxkuminov/obsidian-mcp) with the remote endpoint and the OCI package; Dockerfile carries io.modelcontextprotocol.server.name.
  • registry-publish.yml runs on a v* tag: verifies the tag is on main, retags the CI-proven sha-<short> image as the version, validates and publishes with mcp-publisher over GitHub OIDC. This release is the first publish.
  • ghcr.io/maxkuminov/obsidian-mcp is public from this release; pull :v0.9.3.

Backups exclude derived tables (#372, part of #348)

  • BACKUP_DERIVED_TABLES / --exclude-table-data in the Makefile and the Kubernetes CronJob; BACKUP_FULL=1 escape hatch; restore runbooks updated; docs/architecture/schema-and-migrations.md lists derived vs durable tables.

Dependency hygiene (#370, part of #347)

  • mcp[cli]==1.30.0 stays pinned; Renovate is told allowedVersions: "<2" for mcp (the 2.x line is a separate migration, #365) and docker:pinDigests for base images.

README (#369, closes #345)

  • New first screen, a desktop-local vs headless/remote comparison table dated 2026-10-28, and a paragraph on where Basic Memory and mcp-memory-service fit.

Full changelog: v0.9.2...v0.9.3 (#369, #370, #371, #372, #373, #374)

v0.9.2 — Maintenance: test suite passes on Windows checkouts and outside UTC

Choose a tag to compare

@maxkuminov maxkuminov released this 29 Oct 00:12
5a391ca

Maintenance since v0.9.1. Contributor tooling only: the test suite now passes on Windows checkouts and on machines outside UTC. The server is unchanged.

Upgrade notes

  • Nothing to do. No change to application code, no schema migration, no new settings. Redeploying is optional; the running image behaves identically.

Contributor tooling (#342, contributed by @kelno)

  • Shell scripts keep LF line endings in every checkout. A new .gitattributes (*.sh text eol=lf) stops Git on Windows (core.autocrlf=true) from writing docker/*.sh with CRLF, which bash rejects (set: -\r: invalid option). This made 28 tests fail on such checkouts (test_compose_db_roles.py, test_db_exec.py, test_db_restore.py), and running those scripts by hand failed the same way. The scripts were already LF in the repository, so nothing is renormalized. An existing Windows checkout picks it up after rm docker/*.sh && git checkout -- docker/.
  • A transfer-expiry test no longer depends on the machine's time zone. test_a_naive_expires_at_does_not_raise built its "one second ago" timestamp from local time, while the code reads a naive timestamp as UTC. East of UTC that value was still in the future, so the test failed. It passed only where local time is UTC, as on CI. It now builds the value in UTC.

Full changelog: v0.9.1...v0.9.2 (#342)