v4.3.0

Removed

• Token-saver permissions.deny rules retired; installer now sweeps them away (ADR-0043
  amendment). Daily-use evidence: Read(**/*.lock) hard-blocked reading pubspec.lock in a
  Flutter session (small file, needed to resolve a dependency constraint) and the auto-deny was
  indistinguishable from a manual denial. A prompt-less hard block trades a bounded token cost for
  an unbounded workflow obstruction across heterogeneous projects. configureTokenSaver no longer
  takes a denyRules option; the four old rules live on only as LEGACY_TOKEN_SAVER_DENY_RULES,
  a frozen list that every reconcile (install, --no-token-saver, --uninstall) strips from
  ~/.claude/settings.json while leaving user-authored deny rules untouched (new regression test
  covers the sweep, both with pieces ON and with everything OFF). Token savings on build
  artifacts remain the job of the compaction hooks, which degrade gracefully instead of refusing.

Security

• obscura_research's crawl no longer hands an LLM curator content hidden from every human
  viewer, unfiltered. Found while reviewing Scrapling (a
  Python scraping library) for capabilities worth matching: its own MCP server strips CSS/ARIA-
  hidden elements before conversion, which this package had no equivalent of. Verified live, not
  hypothesized — a synthetic page hiding text behind display:none/aria-hidden had that text
  pass straight through the real obscura binary's --dump markdown (and --dump text, tried as
  a cheaper fix-hypothesis and found to fail identically — neither is visibility-aware) as if it
  were visible, including a prompt-injection-shaped payload that this package's own
  scanInjection heuristic — a visible-text scanner — also failed to flag. Every page
  obscura_research fetches is now parsed with cheerio (packages/obscura-web/src/sanitize.mjs
  — the package's first runtime dependency beyond @modelcontextprotocol/sdk/execa/zod, added
  deliberately for this) and stripped of hidden elements, <template> content, and HTML comments
  before either the heuristic extractor or the local curator sees it. Re-verified live,
  end-to-end, through the real pipeline against the same synthetic payload: the curated result
  contained only real, visible content. Scoped to the automated crawl — obscura_fetch's
  general, single-URL default is unaffected (matching its existing markdown fidelity from
  sanitized HTML would need a second, unapproved dependency); a class-based hiding rule (versus an
  element's own inline style/aria-hidden) is a named, tested, accepted gap, since catching it
  needs a real browser's computed styles. See ADR-0057's third addendum.

Fixed

• CI red since b59e4fb (4 pushes): two independent causes, both reproduced before fixing.
  (1) lint: slop-check.mjs was committed unformatted for the pinned prettier@3.8.4 —
  reformatted (installed copy re-synced). (2) test-python: the ADR-0038 regression test
  (test_usage_boost_decays_stale_credit_so_new_note_not_buried) was green on Windows and red on
  ubuntu — retrieval order was platform-dependent: os.walk returns names sorted on NTFS but
  hash-ordered on ext4, insertion order decides FTS rowids, and rowids are the hidden tie-breaker
  when BM25/cosine scores tie exactly; with a tie at the top, the ranks fed into RRF differ per
  platform and the stale note's residual usage credit (linear taper leaves ~0.022 at 89.5/90
  days) crossed the ~w/(k+r)² one-rank margin, burying the brand-new note the test protects.
  Root fixes, all five ordering sites + the decay: sorted walk in _iter_markdown_files;
  ORDER BY score, path (BM25 FTS query); ORDER BY dist, path, ordinal (sqlite-vec path);
  heapq.nsmallest over (-score, path, ordinal) (brute-force cosine — same O(n·log k));
  (-score, path) tie-breaks in reciprocal_rank_fusion and the post-lever sort; and
  usage_counts_decayed taper linear→quadratic so window-edge credit lands two orders below
  any rank margin instead of inside it. Verified: an adverse-order simulation (reversed
  insertion, the ext4 condition) reproduced the CI failure locally with the ordering fixes alone
  and passes with the full set; 197/197 pytest; all four retrieval/token/assemble bench gates
  exit 0 locally at CI thresholds.
• vkm-design modes/visual-loop.md: capability-check note for preview panes that only render
  file:// inside the project folder. Two real sessions hit the blank "static snapshot" +
  "no site open" wall on an outside-the-project file and misread it as a page bug; the mode now
  prescribes the copy-into-project pattern (dot-named temp, re-copy after each edit, delete at
  loop end) instead of fighting the pane or skipping the loop.
• vkm-design audit scripts: four defects exposed by a real user session log (a session
  "audited" a roadmap HTML with these scripts and half the tool output was wrong — each fix
  below re-verified against the exact input that failed).
  (1) scale.mjs no longer infers the 1.067 ratio: its steps sit 6.7% apart, so the default
  ±3% tolerance bands cover ~91% of the value space and almost any incoherent set "fits" — 22
  real ad-hoc font sizes scored 1 offender under inferred 1.067, a false PASS that buried
  exactly the incoherence the check exists to expose (now honestly 11+ offenders under the
  best non-degenerate ratio, with a hint that --ratio 1.067 remains available explicitly).
  (2) scale.mjs --gen accepts the natural named-flag form (--gen --base 16 --ratio 1.25 --steps 6): the packed-only parser swallowed a following --base as its value and died
  with a misleading "not a comma-separated number list".
  (3) checkSpacing attaches a halfGrid diagnosis when it explains part of the mess: a file
  interleaving 4k and 4k+2 paddings (a real 2px half-step rhythm) was reported flat as "8
  values off-rhythm", sending the session on a manual detour to conclude "false alarm" — the
  CLI now says "only 3 stay off — the rest is a half-step rhythm; judge intent" itself, while
  the exit code still fails against the declared base.
  (4) audit-css.mjs given .html/.htm audits only <style> blocks + inline style=""
  attributes: the whole file used to go through the rule regex, which read a <script> mermaid
  theme object ({ background: "#0e2846", … }) as CSS — one bare color: key away from
  fabricated contrast pairs — while real inline styles stayed invisible. Regression tests for
  all four; installed copies under ~/.claude/skills/ re-synced.
• vkm-doctor skills-drift check: compares EVERY installed skill file, not just SKILL.md.
  The motivating real case (2026-07-18): vkm-discipline/domains/design-ui.md stale under a
  byte-identical SKILL.md — the check reported "ok" for exactly the drift class it was built
  to catch, and the routing fix that file carried (design tasks → vkm-design gate) silently
  never reached sessions. A skill is now STALE if any of its template files differs installed-
  side or is absent (new regression test reproduces the design-ui.md case).
• vkm-design modes/visual-loop.md: new trap — republishing a document to a NEW artifact URL
  while the user still holds the old link. Real session: the fixes landed on a fresh artifact,
  the user's open copy never changed, and the session read back "no veo ningún cambio" as a
  design complaint instead of a delivery failure. The loop now instructs: list existing
  artifacts, redeploy to the SAME url, verify at the link the user actually holds.
• obsidian-memory-rag: concurrent ensure_fresh writers no longer die with database is locked. Two simultaneous vault_hybrid_search calls with a stale index (reproduced live while
  verifying ADR-0056) raced index_vectors' BEGIN IMMEDIATE; the loser only got Python's
  implicit ~5s sqlite3.connect timeout — undocumented and demonstrably too short for a
  batch_commit_every=64 batch under a real embedder. Root-cause fix in the shared
  store.connect(): explicit PRAGMA busy_timeout=30000 on every connection, so a second writer
  blocks and then reads the winner's committed state (the existing mtime/size incremental logic
  already skips re-indexing) — no manual retry loop needed, SQLite's busy handler transparently
  retries BEGIN IMMEDIATE. Verified red→green with a new barrier-synced two-thread test
  (tests/test_concurrent_ensure_fresh.py, slow-embedder holds the lock ~6.4s); full suite 197
  passed, none modified.

Added

• vkm-downloads: background downloads, sets of files, resume, and fastest-mirror selection
  (ADR-0059). Follow-up to the synchronous tool: the user needed to download large files or SETS
  of files "whether the download is fast or slow" and to "find the site with the fastest download" —
  both blocked by the MCP transport's ~60s per-call wall (a 3 GB ISO can't be one synchronous call)
  and by the conservative 500MB cap. Additive (the sync download_file and every ADR-0058 guardrail
  are byte-for-byte unchanged): four new tools + a background job runner. download_start({ files })
  registers a job and returns a job_id immediately, running the transfer as an async task inside
  the long-lived stdio server — no child process, so the "never execute downloaded bytes" property
  holds; download_status / download_cancel poll and abort. Each file may list several mirror
  URLs; probe_mirrors (and prefer_fastest) measures each with a ~512 KB ranged GET and ranks by
  real throughput (latency tiebreak; a size_mismatch flag catches a "mirror" that isn't the same
  file), all through the same guarded, IP-pinned path so a private/loopback mirror is refused, not
  probed. Large files are gated by a free-disk-space check (fs.statfs) rather than a hard byte
  wall (background ceiling VKM_DOWNLOAD_MAX_BYTES_BG, 100GB); interrupted downloads resume from
  their .part via HTTP Range. Verified live end-to-end against the real Hiren's BootCD PE servers:
  probe_mirrors ranked its two real mirrors by measured speed (edgeuno 9.1 Mbps vs origin 8.8 Mbps,
  same 3,291,686,912-byte size), and the 3 GB ISO the sync path refused streamed in the background at
  ~130 Mbps and cancelled cleanly (.part removed). 18 new tests (mirror ranking with an injected
  clock, the job runner over injected request/DNS incl. batch/mirror-pick/cancel/resume, the disk
  guard, and the four MCP handlers).

• vkm-downloads: a guarded file-download manager MCP, opt-in via --downloads (ADR-0058).
  The user wanted Claude to download files on request (find a URL, save it to disk) — a capability
  no existing tool had, and one that writes bytes to disk rather than reading the web as DATA. New
  self-contained package @vkmikc/vkm-downloads exposing download_resolve (metadata only — HEAD/
  GET-and-abort, writes nothing, safe to call before asking the user) and download_file (streams
  to ~/Downloads/vkm-kit/, returns path/size/SHA-256). Its own installer flag, deliberately NOT
  in --full (even though --full includes --obscura): opting into stealth web reads must
  never silently grant disk writes — consent stays granular. Guardrails, all tested against an
  injected fake request/response + fake DNS (no sockets, no real DNS): http(s) only; the host is
  resolved once and refused if any record is loopback/private/link-local/CGNAT, with the socket
  pinned to the validated IP (node:http(s) + a custom lookup, chosen over fetch precisely
  to close the DNS-rebinding TOCTOU that a resolve-then-fetch leaves in that guard — the guard that
  stops an injected URL from targeting the kit's own loopback services like Ollama :11434);
  redirects followed and re-validated per hop (a 302 to 127.0.0.1 is refused); a 500MB cap
  (VKM_DOWNLOAD_MAX_BYTES) enforced by Content-Length and mid-stream, deleting the partial
  file on abort; filenames sanitized (../, separators, %2f, NUL, Windows-illegal + reserved
  device names) with a path.resolve root-check before writing and no-overwrite de-dup;
  extension from the URL/Content-Type, never an unvalidated Content-Disposition. Downloaded files
  are never opened or executed (no code path spawns a process — grep-verifiable), and every
  download is appended to a JSONL audit log (~/.vkm/downloads/downloads.log). The tool descriptions
  steer the agent to resolve-then-confirm-then-download, reinforcing (not replacing) Claude Code's
  own per-file download-permission rule. Version-locked to the kit (ADR-0042/0051). 38 new tests.

• obscura_research: the answer was being lost at gather, not at rank (ADR-0057). User report:
  "search too little, nothing removes the junk, Claude gets the leftovers." A bench built first and
  gated on (evals/research/, metrics ported from obsidian-memory-rag's bench_recall.py) falsified
  three assumptions in the approved plan before any pipeline code changed — most importantly, that the
  BM25 rerank ADR-0054/0055 shipped was actively worse than doing nothing: SearXNG already fuses rank
  across every engine that returned a URL (results.py#calculate_score), and re-ranking survivors with
  BM25 over ~30 tokens of SEO snippet threw that away for -0.115 nDCG@10. The rerank is deleted, not
  tuned. Separately, the vocab-mismatch case ADR-0055 was built for turned out to be a GATHER failure,
  not a ranking one — the answer was never in the pool to begin with (a colloquial Spanish query
  returned Instagram reels and Facebook posts; the same question in canonical English technical wording
  returned the actual OWASP page at rank 4) — so query expansion, not reranking, is what fixes it.

  • Pipeline: serp.mjs preserves SearXNG's own order/score instead of re-ranking (rankCandidates
    stays exported for the no-SearXNG fallback only); fetch runs at bounded concurrency (semaphore,
    default 4) instead of sequentially; top_k default 8→20 (calibrated for the old sequential loop,
    now the knob deciding whether an expanded, fused pool's answer is read at all); a wall-clock deadline
    (OBSCURA_RESEARCH_DEADLINE_MS, default 50s) returns the best-ranked finished prefix with
    partial/remaining rather than losing everything to the MCP SDK's non-negotiable 60s
    tools/call timeout (resetTimeoutOnProgress defaults false — the server cannot extend it).
  • Query expansion (expandQuery, ollama-client.mjs) turns one question into several
    canonically-worded, translated sub-queries, run against SearXNG's general,it,science categories
    (it/science are site APIs — github, mdn, arxiv, semantic scholar — that answer while
    Google/Brave-backed general scrapers are suspended, and don't themselves get suspended; costs zero
    extra upstream requests, one /search call per sub-query). Runs on a separate, more capable model
    (OBSCURA_OLLAMA_EXPAND_MODEL, default qwen3.5:4b-q4_K_M) from curation's — expansion is a recall
    task (does the model already know "hidden instructions on a page" is called prompt injection?), not
    reading comprehension, and measured phi4-mini at +0 answers found while doubling social-media junk
    in the pool. Absent that model, the tool correctly does NOT expand rather than expand badly.
  • The curator (curatePage) now returns a banded relevance: 0-10 + reason instead of one bit, and
    drops results below a conservative threshold (dropped reported, include_rejected: true
    escapes it) — reversing ADR-0055 §6, which returned relevant: false results anyway.
  • Ban-avoidance, found the hard way (fixture capture + probes suspended every SearXNG-backed engine
    for 180s-3600s, and a suspended instance answers HTTP 200 + results: [] — indistinguishable from
    "no hits" unless unresponsive_engines is read): sub-query fan-out throttled
    (OBSCURA_SUBQUERY_CONCURRENCY, default 2); a SearXNG response cache keyed by
    (query, page, categories) that distinguishes a stable "exhausted" empty from a transient "banned"
    one (only the former is cached); enginesUnavailable surfaced in the response so "no such page
    exists" reads differently from "come back in 3 minutes"; bench-capture.mjs refuses to overwrite
    good fixtures with empty ones captured against a rate-limited instance.
  • Resilience hardening adapted from Scrapling (verified
    against its actual source, not its README): its "adaptive scraping" is SequenceMatcher.ratio()
    scored across DOM element facets (tag/text/attributes/tree path/siblings) — this package has no DOM
    to fingerprint that way (SERP records are flat, pages are markdown), so only the primitive itself
    ported (text-similarity.mjs#similarityRatio, a from-scratch port pinned against difflib's own
    canonical example). Used by a new generic, similarity-ranked SERP fallback
    (serp.mjs#genericExtractLinks) engaged when a specific per-engine HTML parser breaks on markup
    drift — the exact risk ADR-0051 already flagged and had no mitigation for beyond fixtures. Also
    added: pragmatic robots.txt compliance (robots.mjs, fail-open, scoped to the automated crawl only,
    never obscura_fetch) and a page-fetch cache mirroring the search cache one stage later in the
    pipeline. TLS fingerprint impersonation and proxy rotation were both explicitly declined (the first
    is redundant with obscura's own real-browser rendering; the second is out of scope for a local,
    personal-use tool with no measured need).
  • Bugs fixed in passing: extraction: "ollama" no longer lies when the excerpt is empty or the
    verdict is false; a failed fetch reports relevant: null instead of hardcoding true (it used to
    outrank a page the curator had actually rejected); truncated propagates instead of being silently
    dropped; capResults bounds the response (the prior worst case, ~30k tokens, exceeded
    MAX_MCP_OUTPUT_TOKENS's 25k default); one URL identity (url-identity.mjs, with a frozen,
    tripwire-tested hash8 so RESEARCH/'s existing notes are never orphaned) replaces three
    independently-drifted ones.
  • Mass research now accumulates across calls instead of re-covering the same ground:
    persist:true + topic reads RESEARCH/<topic>/sources/ (already a durable, one-file-per-URL
    record) before crawling and excludes already-banked candidates, so top_k's budget goes to
    genuinely new pages on a second call — alreadyCovered reports how many were skipped.
  • Fixed a real deadline bug, found only by running the redesigned pipeline live against a
    real SearXNG + Ollama + obscura stack: a call configured with a 50s deadline took 58.3s
    wall-clock, within 1.7s of the MCP transport's hard 60s cutoff. The concurrency pool's deadline
    check only gated picking up new work; an item already in flight ran to its own independent
    timeout (obscura's fetch ~45s worst case, curation's 30s) instead of the deadline meant to
    bound the whole call. Each item is now raced against the time actually left, marked
    timedOut: true if it doesn't finish — re-run live with identical parameters: 58.3s → 50.012s.
    No mock-based test caught this; it took a real run against real services.
  • Mass research now genuinely accumulates across sessions, not just calls: persist:true +
    topic reads RESEARCH/<topic>/sources/ (a durable, one-file-per-URL record) before
    crawling and excludes already-banked candidates via excludeHashes — a second call on a
    well-covered topic, even from a fresh MCP process days later, reaches further into the pool
    instead of returning the same top hits (alreadyCovered reports how many were skipped).
  • A second real deadline gap, found by tracing every caller after the first fix: the
    SearXNG gather phase's sub-query fan-out never received the deadline at all (an oversight,
    not a design gap — mapWithConcurrency already supported it), so a degraded SearXNG could
    burn the whole budget gathering candidates before a single page was ever fetched. Fixed the
    same way — the deadline now gates both the outer fan-out and the inner per-sub-query page
    walk — with two new regression tests.
  • packages/obscura-web 239/239 tests. New ADR-0057 (supersedes ADR-0055 §5/§6, reverses
    ADR-0054's sequential-fetch choice). Verified live end-to-end: a real query against the
    real stack expanded into 5 genuine sub-queries, curated sqlite.org/lockingv3.html and
    sqlite.org/wal.html into its top results (relevance 9 and 8), and persisted 8 curated notes
    to disk as real files. obscura_fetch gains an opt-in css_selector param and a new sibling
    tool, obscura_fetch_many (bounded-concurrency batch fetch, one URL's failure doesn't sink
    the rest) — both inspired by reviewing Scrapling's own MCP server tool surface. Full monorepo
    green running the actual CI command (npm run test --workspaces --if-present) from the repo
    root, not just this package in isolation.
  • Quality-based early stop: target_relevant. Left at its default, a call always spent its
    whole top_k/deadline_ms budget even after the curator had already confirmed plenty of good
    pages — the tool had no notion of "enough", only "out of time" or "out of candidates". Set
    target_relevant, and the call stops dispatching further candidates as soon as that many pages
    have been genuinely curated relevant (a real curator verdict at/above drop_below; a heuristic
    fallback or an unfetched page never counts), returning targetReached: true so a short response
    reads as a deliberate success rather than a cutoff (partial/remaining can still report
    alongside it — the two are not exclusive).
  • Fixed a real, if narrow, relevant/drop_below inconsistency found building the above:
    curatePage has no dropBelow parameter and derived its own relevant boolean from the
    module's fixed default, so a call using a non-default drop_below could get a per-result
    relevant:true that quietly disagreed with that very call's kept/dropped filter (relevance
    5 with drop_below:7 reported relevant:true against curatePage's baked-in 3, even though the
    result was correctly dropped). relevant is now derived once, in research.mjs, from THIS
    call's own dropBelow — byte-identical for callers using the default. One stale test asserted
    the pre-ADR-0057 "not-relevant still returns a result" behavior that was deliberately reversed
    above; rewritten to exercise the same extraction correctness through include_rejected
    instead. packages/obscura-web 246/246 tests.
  • The curator bake-off fase 3 was approved on (measure, don't assume the model), run — with an
    honest, inconclusive result. Built the scorer that didn't exist (bench-curate.mjs +
    a real, labelled golden set anchored on the exact live MDN-false-positive failure above) and
    ran phi4-mini:3.8b-q4_K_M (current default) vs qwen3.5:4b-q4_K_M vs qwen3.5:9b-q4_K_M
    as curator. qwen3.5:9b is the only one that stopped scoring the MDN pages relevant (0 false
    positives) but at 40.6s/page with half its calls timing out at 60s it is disqualified by
    latency alone (removed via ollama rm, per this session's own pre-declared bake-off rule).
    Between the two practical options, qwen3.5:4b does not beat phi4-mini (lower accuracy,
    identical false-positive rate on the hard case, slightly slower) — default stays
    phi4-mini, and the false-positive failure itself remains open at this model size class on
    this hardware (a rubric fix — the prompt's own "official documentation" band plausibly
    rewarding MDN's general authority over its topical relevance — is the next unexplored lever,
    not attempted this round). n=8 declared small, matching this ADR's existing honesty about the
    expansion bake-off's own sample size. DEFAULT_MODEL (ollama-client.mjs) now carries this
    reasoning inline, matching the doc-comment DEFAULT_EXPAND_MODEL already had for its own
    bake-off — the decision used to live only in the ADR, not at the line of code it governs.
  • Fase 4, partially shipped: embeddings wired for near-dup removal + MMR diversity; per-page
    chunk selection built but NOT wired. ollama-client.mjs gains embedPassages
    (OBSCURA_OLLAMA_EMBED_MODEL, default qwen3-embedding:0.6b, batched /api/embed) and
    rank.mjs gains mmr. Both wired into deepResearch as two new opt-in params sharing ONE
    batched embed call over the final kept set (never per-page): dedupe_similar (drops a later
    page whose embedding is near-identical, cosine ≥0.92, to one already kept — an objective
    redundancy signal, never a relevance judgment; count reported as dedupedSimilar) and
    diversify (MMR reorder, mmr_lambda default 0.5, off by default per ADR-0028's own
    precedent). Both skipped when too little deadline_ms remains or the embed model isn't
    pulled, degrading silently to the plain curated order. chunk.mjs (sliding-window,
    paragraph-respecting, overlapping chunking) is also built and tested, but deliberately left
    UNWIRED into the per-page fetch+curate path — that would need per-page embedding calls inside
    the deadline-critical hot loop, and there is no bench measuring whether chunk-based passage
    selection actually beats today's blind 12k-char truncation; wiring something unmeasured into a
    hard-deadline path is the wrong trade, matching this session's own repeated "a phase that
    doesn't move the metric doesn't merge" discipline. packages/obscura-web 283/283 tests.
  • Both remaining gaps attempted directly — both honestly negative, neither shipped. A
    domain-check revision to CURATE_SYSTEM_PROMPT (name the query's and page's technology; cap
    the score if they differ) was measured over 3 trials vs 2 on the baseline: false-positive rate
    improved inconsistently while false-negative rate got WORSE — the same instruction that
    correctly rejected an MDN page in one trial also made the model reject sqlite.org/wal.html
    itself as off-topic in two of three, the very vocab-mismatch failure ADR-0055 exists to
    prevent. Reverted; the finding lives in the prompt constant's own doc-comment.
    selectChunksByRelevance (chunk.mjs) — rank chunks by similarity to the query, keep the
    best up to the input budget — was tested live against two real long pages with a verified
    fact buried past the 12k-char truncation point: one case improved (relevance 5→8, on-topic
    reason emerged), one regressed (relevance 8→5 — greedy similarity selection discarded a
    page's own orienting introduction that blind truncation keeps for free by being first).
    Not wired. Both negative results are recorded in ADR-0057's seventh addendum in full, not
    softened — a negative result this session already paid to learn is worth more than a
    positive one it didn't earn. packages/obscura-web 288/288 tests.

• vkm-doctor: skills-drift check. Real gap found on the maintainer's own machine: vkm-design
  and vkm-research existed in the kit's templates but were missing from ~/.claude/skills/
  (install predated their addition) and nothing noticed. The doctor now compares what the kit
  defines against what's installed and reports MISSING (no SKILL.md installed) and STALE
  (installed SKILL.md content differs from the template), with an actionable reinstall hint. The
  canonical list comes from create-vkm-kit's own skillAssetFiles resolver via a dynamic import
  isolated in src/skills-drift.mjs — it can never drift from what the installer actually installs,
  and when that import isn't resolvable (standalone npm install) the check degrades to an honest
  skip, never a silent false-negative; no ~/.claude/skills/ directory at all is likewise an
  informative skip. packages/vkm-doctor 11/11 tests green.

• RESEARCH/: a persistent web-research knowledge bank in the same vault (ADR-0056). Research
  passages from obscura_research (ADR-0054/0055) used to die with the tool's response; this makes
  them accumulate. Spans three packages plus the installer/docs:

  • packages/obscura-web: obscura_research gains opt-in persist: boolean (default false) +
    topic (slug ^[\w][\w-]*$); with persist it returns persisted: {topic, written, updated, dir} and writes one verbatim-extract note per curated result to
    RESEARCH/<topic>/sources/<hash8-url>-<slug>.md (frontmatter: url/title/retrieved/query/
    extraction/relevant/origin: web/status: raw), plus a per-topic hub (_index.md: query log,
    open questions) and a regenerated global index (RESEARCH/_index.md). Requires
    OBSCURA_RESEARCH_DIR (typed error if unset); every write is root-checked against it first.
    Dedup by URL hash8 updates in place, never duplicates; fetchFailed results are never
    persisted, relevant:false ones are (flag intact). persist:false (default) is byte-identical
    to the pre-ADR-0056 tool. New tool obscura_consolidate(topic, force): the local-Ollama half of
    dual consolidation — map-reduces sources/ (≤12k-char batches) into summary.md,
    status: draft-local; a consolidated summary is never overwritten (force or not); no Ollama
    → typed OllamaUnavailableError, no silent fallback. src/research-persist.mjs (new),
    src/ollama-client.mjs gains chatJSON/summarizeNotes. packages/obscura-web 87/87 tests
    green.
  • packages/obsidian-memory-rag (196 pytest) + packages/obsidian-memory-mcp (164 node): retrieval-level
    isolation so folder convention alone doesn't leak research noise into memory recall.
    paths.py adds RESEARCH_PREFIX/validate_section/in_section/section_sql_filter;
    search_vault/hybrid_search gain section: "research" | "memory" | None that cuts at
    candidate collection (BM25 + vector), filters graph neighbors, and re-applies the invariant on
    the final fused list (query.py:699-703) — None stays byte-identical. CLI gets --section
    with choices on all four search subcommands. vault_fts_search/vault_hybrid_search mirror
    section (zod enum); assemble_context gains include_research (default false → passes
    --section memory). MCP schema-budget gate (10,800 chars total / 450 per string) stays green,
    landing at 10,748.
  • packages/create-vkm-kit: the generated obscura-web MCP config gets
    OBSCURA_RESEARCH_DIR = <vault>/RESEARCH by default (override: --obscura-research-dir).
    New skill /vkm-research (fourth vkm skill): consolidates a topic's sources into a quality
    summary.md (wikilinks, supersedes on contradiction), marks the hub, and doubles as the
    import path for hand-authored reports dropped into sources/. Vault scaffolding seeds
    RESEARCH/_index.md and links it from START_HERE.md's vault map (no orphan-at-birth). Managed
    CLAUDE.md/AGENTS.md block gains a short es/en "Investigación/Research" rule (research recall via
    section:"research", memory recall stays uncontaminated, the memory-close ritual never writes
    under RESEARCH/, origin: web notes are untrusted data) — budget raised as a reviewed
    decision (ADR-0036 precedent): es 8,660→9,375 chars (budget 9,100→9,850), en 8,421→9,093 chars
    (budget 8,850→9,550), ~5% headroom kept. packages/create-vkm-kit 242/242 tests green (1
    unrelated skip).
  • Docs: ADR-0056, docs/{es,en}/glosario.md/glossary.md (RESEARCH/, obscura_consolidate,
    the raw → draft-local → consolidated lifecycle), packages/obscura-web/README.md.

• obscura_research: deep web research as local CPU/RAM work, not tokens (ADR-0054). Third tool
  in packages/obscura-web, alongside obscura_fetch/obscura_search. Origin: a user pushback that
  obscura_search's 20-result ceiling was too shallow for real research, and that the fix must not be
  "dispatch a subagent" — a subagent still spends real LLM tokens/quota, it only relocates the cost out
  of the main conversation. The actual lever: obscura/SearXNG already run as local, per-request/
  on-demand OS processes (ADR-0051/ADR-0052) that cost zero tokens to execute; the gap was that the
  "search deep" loop lived in the agent calling the MCP N times. obscura_research(query, max_candidates=50≤300, top_k=8≤30, passage_chars=800) closes that gap: (1) walks SearXNG's pageno
  server-side (searxngSearch extended with a page param, verified live against
  docs.searxng.org/dev/search_api.html) to gather up to max_candidates — one loopback HTTP
  round-trip per page, zero LLM tokens; (2) ranks every candidate's title+snippet locally with BM25
  (rankCandidates, k1=1.5/b=0.75, +1-smoothed IDF so a small pool never scores negative) — the
  candidate pool is its own corpus, no external index or embedding call; (3) fetches only the top
  top_k via the existing obscuraFetch; (4) excerpts each to the paragraph(s) that actually match the
  query (extractPassage) instead of returning the whole page. Response carries scanned (candidates
  gathered) and fetched (pages read) alongside results, so the agent sees how deep it searched
  without paying tokens for the difference. Honest degrade: no local SearXNG → skips the pageno loop
  and falls through to searchWeb's existing single-page scrape chain (same ceiling as
  obscura_search) rather than hand-rolling fragile per-engine offset pagination against
  DuckDuckGo/Bing/Brave HTML — serp.mjs already documents why that path doesn't scale. Zero new npm
  dependencies. 47/47 @vkmikc/obscura-web tests green (11 new), 49/49 elsewhere in the monorepo,
  unchanged.

• obscura_research curates pages with a local Ollama model instead of keyword overlap
  (ADR-0055). Follow-up user pushback on the above: BM25/keyword-window extraction alone
  systematically discards exactly what research is supposed to find — information that's related
  to the query but doesn't share its vocabulary. Fix, per the user's own spec: a local LLM must
  understand and relate each page, one page at a time, nothing accumulated in context, only the
  already-curated info handed back. packages/obscura-web/src/ollama-client.mjs — a
  self-contained adaptation of vkm-spec/src/ollama-client.mjs's already-proven pattern (typed
  OllamaUnavailableError, structured-output format, deterministic-fallback hard invariant;
  read in full before reuse, not assumed), same default model (phi4-mini:3.8b-q4_K_M) and host
  (127.0.0.1:11434) as vkm-spec so both share one daemon. deepResearch now runs one
  checkOllama per call (not per page); when available, each of the top_k fetched pages is
  curated individually via the new curatePage(markdown, query) (input capped at 12,000 chars,
  truncated reported, never silent) before its full text goes out of scope — never combined
  with another page's text, never returned to the agent, only the curated excerpt is. A single
  page's curation failure degrades only that page to the ADR-0054 heuristic (now itself improved:
  extractPassage keeps the paragraph after a keyword match too, since elaboration rarely
  repeats the matched term) — one flaky page never sinks the whole call, and the tool keeps
  working with Ollama absent exactly like it keeps working with obscura/SearXNG absent. Every
  result now reports extraction: "ollama" | "heuristic" and relevant: boolean. New env vars,
  all optional: OBSCURA_RESEARCH_OLLAMA=0, OBSCURA_OLLAMA_HOST, OBSCURA_OLLAMA_MODEL,
  OBSCURA_OLLAMA_KEEP_ALIVE. Zero new npm dependencies (reuses zod). 66/66
  @vkmikc/obscura-web tests green (19 new).

• obscura_research final result order now trusts Ollama's verdict over BM25. Found via a
  real (non-mocked) end-to-end run against "bees": chosen's fetch order comes from BM25, which
  is blind to meaning, but the response kept that same order even after a page was actually
  curated — a page Ollama explicitly marked relevant: false could still outrank one it
  confirmed true, purely because BM25 scored the rejected page's title/snippet text higher.
  deepResearch now re-sorts results by relevant (stable — ties keep their original BM25
  order) after curation. 67/67 @vkmikc/obscura-web tests green (1 new, reproduces the ordering
  bug with a keyword-stuffed-but-off-topic page BM25 would otherwise rank first).

• Write-time hygiene lint (vault-lint.mjs): drift now dies at the write path. A manual
  hygiene pass on a real vault (2026-07-12) found three classes of silent drift no tool prevented
  at the source: 17 observation lines with non-canonical categories ([DECISIÓN], [LECCIÓN GENERAL], [regla] — invisible to vault_observations(category:'decision')), 6 broken
  [[wikilinks]], and 18 orphan notes the graph couldn't reach. Every vault_write_file /
  vault_edit_file / vault_append_file now returns warnings when the text being written has
  (a) a non-canonical observation category (suggests the canonical one; per-vault extras via
  memory-schema.json → observationCategories), (b) a [[wikilink]] that resolves to no note
  (template placeholders <…> exempt), or (c) a NEW knowledge note (PROJECTS/STACKS/PRACTICES/
  RULES) with zero links — orphan at birth. Strictly scoped to the text the call introduces
  (never nags about pre-existing lines), warning-only (never blocks), fail-open (a lint crash
  cannot break a write), and zero schema cost (no new tools, no description growth).

• Scaffolded vaults are born connected. create-vkm-kit's START_HERE.md now ships a
  "Vault map / Mapa del vault" section wikilinking every note the scaffold creates (MEMORY,
  SESSION_LOG, PRACTICES/*, RULES/TEMPLATE, _meta/agent-profiles) — the hub whose absence let
  the 18-orphan drift accumulate.

• vault_audit no longer flags template placeholders. [[PROJECTS/<proyecto>]] in
  RULES/TEMPLATE is deliberate scaffolding; the broken-link scan now skips <…> targets — the
  same exemption the write-time lint applies.

• /vkm-design skill: professional, anti-generic design for any medium (ADR-0053). Third vkm
  skill installed by create-vkm-kit. A designer-cognition core (references/designer-mind.md:
  Gestalt, hierarchy-first, process order, crit vocabulary), a mandatory design-direction protocol
  with a banned-by-default "slop fingerprint" (references/direction.md), quantified foundations
  (type scales, OKLCH palettes, motion numbers, inline WCAG formula), a live-verified library map
  across web/mobile/desktop/TUI/dataviz (references/libraries.md — candidates unversioned, hard
  verify-online rule), four modes (generate, critique, visual-loop, handoff with DTCG
  tokens), executable lineage capsules (references/lineages.md — real free typefaces + OKLCH
  seeds + shape/motion numbers per design language), a fully worked brief→deliver example with
  real validator output (examples/worked-example.md), ceremony scaling (full protocol for new
  surfaces, inherited direction for edits, numbers-only for micro-tasks), and four zero-dep
  validators with CLI exit codes and unit tests: scripts/contrast.mjs (WCAG; accepts hex AND
  oklch() per CSS Color 4 with gamut-clip detection), scripts/scale.mjs (check + --gen),
  scripts/palette.mjs (gamut-aware OKLCH neutral/accent ramps + semantic ok/warn/danger tokens
  searched to a guaranteed >= 4.5:1 on both surface extremes; --json emits DTCG tokens) and
  scripts/audit-css.mjs (static stylesheet audit for critique mode: declared color pairs with
  var() resolution, font sizes, spacing rhythm — honest static-only scope), plus
  scripts/slop-check.mjs (the anti-generic fingerprint mechanized: scans HTML/CSS for
  Inter/Poppins defaults, the indigo/violet family, gradient text, glassmorphism, emoji
  iconography, uniform radii and stock shadows — exit 1 until each hit is justified or
  replaced; measured end-to-end in evals/design-bench/). Critique mode also
  gains an "infer the incumbent direction" workflow for edits inside existing systems, and
  references/marks.md covers logo/wordmark/favicon craft (construction, optical corrections,
  16px→512px test matrix, SVG hygiene). After user feedback that a gate-passing page can still
  be forgettable, references/contemporary.md adds the ceiling: time-stamped award-level
  currents (2026, verified online) as executable recipes with their slop-versions, a "boldness
  budget" (1–2 full-intensity moves per surface, in the first viewport) and the lineup test
  (logo hidden, distinguishable from ten templates?) wired as a Major finding in critique, a
  per-iteration check in the visual loop and a hard requirement of the committed direction —
  correct is the floor, memorable is the target. references/illustration.md handles any bespoke
  figurative drawing of a real thing (species, product, landmark, mascot): because hand-plotting
  bézier coordinates cannot depict a complex real subject faithfully — a model emits the category
  average, so a hand-drawn "guapote" cichlid came out a tuna with every gate green — it teaches a
  technique-matching decision (trace a reference / treat a photo / icon library / hand-draw only if
  abstract), a real tracing pipeline scripts/trace-svg.mjs (potrace + jimp: reference → Otsu/
  alpha mask → vector → restyle), and an IoU fidelity gate (measured overlap of trace vs
  reference, ship ≥ ~0.7). Proven on evals/design-bench "The Cabinet": 7 subjects across 3
  categories traced from CC/CC0 references, IoU 0.66–0.92. examples/illustration-gallery.md runs
  the technique choice across 8 subjects; wired as a Major finding in critique/visual-loop and a
  build step in generate. illustration.md Step 4 + foundations.md add the mixed-ratio
  alignment rule: logos/avatars/icons/illustrations of different native aspect ratios in one
  grid render at different sizes and off-centre (a "desfase") — normalize each to a shared box by
  its longest side (object-fit: contain for raster; shared square viewBox +
  preserveAspectRatio for SVG), with trace-svg.mjs --square producing grid-ready output at
  trace time. trace-svg.mjs also cleans the mask before tracing (largest connected
  component + fill-holes + morphological smoothing) so the trace follows the subject's shape, not
  the photo's background grain and interior texture — a raw threshold traces the noise and ships
  a speckled artifact; a cleaned one traces to a single smooth path. Documented limits (both
  caught by the IoU gate): a busy background can't be thresholded, and fill-holes closes
  meaningful holes — so the rule is trace CLEAN references, verify, and drop what won't come out
  faithful rather than ship artifacts. scripts/treat-photo.mjs (jimp-only) covers the table's
  other faithful branch — treat a real photo with duotone / halftone / cutout when a subject
  has no clean line to trace (fine texture, a portrait): the output IS the photo so it stays
  faithful with full detail, the treatment makes it one system, no trace artifacts. The
  illustration toolkit now matches the decision table end to end: trace a clean reference OR treat
  a photo — both faithful, neither hand-plotted (evals/design-bench run 6f). The lineage
  catalog quadrupled (8 → 16 executable capsules: naturalist/field-guide, heritage/workwear, Y2K
  chrome, groovy 70s, sci-fi HUD, Scandinavian folk, collage/zine, soft-dimensional/clay) and the
  contemporary currents grew 10 → 14 (image-treatment-as-identity, sticker/badge layer, giant
  cropped wordmark, custom cursor/selection). Final polish from live A/B feedback: the skill
  declares no house style (a model's second-order comfort look — one favourite type pairing/
  colour world reused across briefs — is named in the slop fingerprint), the variety sampler
  forces rotation on five axes (hue, lineage ≠ last build, type pairing, current, layout
  topology — the reused header→hero→grid→pricing→giant-wordmark skeleton is called out as
  structural convergence), and foundations.md adds "when the image IS the content, show it
  WHOLE" (natural ratio / contain, never a cover-crop gallery). vkm-discipline's
  domains/design-ui.md stays the acceptance gate and now points to the skill.

• vault_delete_file + vault_move_file: note lifecycle lands in the hybrid MCP (17 tools).
  Until now the vault surface could create and edit notes but never retire or reorganize them —
  deletes/renames meant dropping to the shell (outside the vault lock, no path-escape checks, no
  etag precondition). Both tools run under the advisory write lock with safeVaultPath validation
  and opt-in ifMatch. Safety rails, chosen so an agent mistake is recoverable and a broken link is
  visible instead of silent:

  • Delete is soft by default — the file moves to .trash/<same relative path> (Obsidian's
    vault-trash folder, uniquified on collision); permanent: true unlinks for real and is the only
    way to remove something already in .trash/. Restore is just vault_move_file back out.
  • Move refuses an existing destination without overwrite: true, creates destination parent dirs,
    and runs the destination-side memory-schema.json check like a fresh write there would.
  • Both refuse directories and the core protocol notes (START_HERE/MEMORY/SESSION_LOG/
    KNOWN_FAILURES), and neither rewrites [[wikilinks]] — the result reports which notes still
    reference the old name (linkRefs/staleLinkRefs, boundary-checked so [[typescript-advanced]]
    never counts for typescript) so the agent fixes links deliberately.
  • Schema budget (ADR-0035) raised 8,000 → 9,200 chars in the same change: two destructive-capable
    tools whose rails must be visible in the schema, not discovered by error. Also fixes the stale
    "fourteen tools" count in the stack rule (it was 15 — assemble_context was never enumerated).

• Note-lifecycle tool set completes the hybrid MCP surface (22 tools): vault_append_file,
  vault_frontmatter_set, vault_backlinks, vault_git_history, vault_rotate_log.

  • vault_append_file — CRLF-aware append (creates if missing, normalizes newlines to the file's
    EOL, guarantees separation): the close ritual's SESSION_LOG one-liner no longer needs a read +
    single-line-anchor edit round-trip.
  • vault_frontmatter_set — set/remove top-level scalar YAML frontmatter keys without
    text-matching (status: hypothesis→confirmed, last_verified). Creates the block when absent,
    preserves everything else byte-for-byte, refuses nested/multi-line values instead of corrupting
    them, JSON-quotes values YAML could misread.
  • vault_backlinks — read-only "who [[links]] here" (boundary-checked, self-references
    excluded); the impact check before a delete/move, and it works for already-deleted targets.
  • vault_git_history — read-only bridge to the sync repo: a note's commits, or its content at a
    rev (strict hash/HEAD~N allowlist, never parsed as a git option) — recovery even after
    permanent: true. Old content returns inside the untrusted-data envelope like any vault read.
  • vault_rotate_log — MCP face of the existing rotate-log engine via a new Python
    json-rotate-log subcommand (same options, one JSON object out): archive old SESSION_LOG.md
    sections keeping the newest N; moves, never deletes; dryRun previews. Closes the loop with
    vault_audit's bloat warning in-session.
  • The rotate engine (rotate.py) now also rotates flat bullet logs: when SESSION_LOG.md
    has no ## sections, top-level - bullets (the close ritual's "one line at the end" format,
    indented continuations attached) are the rotation unit; sections still win on mixed files so a
    bullet list inside a section is never rotated on its own. RotateResult/json-rotate-log gain
    a mode: "sections" | "bullets" field. Found live: the shipped vault convention writes bullet
    lines, so section-only rotation was a silent no-op (sections_total: 0) on real logs.
    vault_memory_report's recent-activity digest (reflect.py) shares the splitter and inherits
    the fallback — a flat bullet log now counts its newest entries instead of reporting zero activity.
  • Schema budget: 9,200 → 10,800 chars (measured 10,622) — five tools at ~320 chars each, already
    trimmed to the load-bearing contract.

• npm run preview:assets — local preview server for theme-aware SVG assets. Tiny
  dependency-free static server (scripts/preview-assets.mjs, 127.0.0.1:4180) for visually
  verifying docs/assets/*.svg in a live browser, where GitHub's prefers-color-scheme theming
  actually applies. Exists because agent browser panes commonly block file:// navigation (hit
  live while QA-ing the hero update). Read-only, GET/HEAD only, localhost-only, traversal-safe,
  never serves dotfiles. Documented in CONTRIBUTING.md.

Changed

• Docs refresh + restyle across the repo (docs + --help text only, no runtime change).
  Root README.md/README.en.md: npm / node / platform badges + an at-a-glance suite strip.
  docs/{es,en} glossaries: the ten missing 4.x terms added (vkm-kit, token-saver,
  vkm-doctor, vkm-spec, skills, assemble_context, knowledge graph, memory report,
  obscura-web, SearXNG). docs/observability.md: the vkm-doctor surface (local OTLP sink on
  127.0.0.1:4319 → ~/.vkm/telemetry/, labelled --include-transcripts fallback) was entirely
  undocumented — now surface #2, summary table included. packages/create-vkm-kit/README.md:
  stale Vahlame/obsidian-memory-kit repo URLs fixed, suite paragraph added, Options table
  completed with the twelve missing flags (--minimal, --pin-failures/--usage-boost,
  --memory-enforcement, --effort-gate, --token-saver/--terse-style, --telemetry,
  --skills/--agents, --ollama, --uninstall). New packages/vkm-doctor/README.md and
  packages/vkm-spec/README.md (neither had one). docs/README.md hub: usage / migration /
  glossary links. --help drift fixed: the skills paragraph now lists /vkm-design (ADR-0053)
  and says "the three skills" (was "the two"). docs/assets/hero.svg: new bottom band "Suite de
  eficiencia 4.x" (token-saver, vkm-doctor, vkm-spec, skills, obscura-web) — the hero only told
  the 3.x memory story; same light/dark theming, new accent contrast-checked (5.45:1 light /
  6.95:1 dark), README alt texts updated to match.