Skip to content

v1.2.0

Latest
Latest

Choose a tag to compare

View all tags
@ethanj ethanj released this 10 Sep 02:52
v1.2.0
84ed65c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

[1.2.0] - 2026-09-09

Highlights

  • Profile-aware viewer navigation, declared record fields, relationships, provenance and evidence drill-down. Stored source previews are distinguished from original-publication links; artifact bytes are freshly verified and served only on loopback. Narrow layouts, plain-language labels and initially fitted, yielding graph layouts improve browsing.
  • OrcaRouter provider support with openai/gpt-4o-mini as its default and explicit credentials; embeddings remain unwired and fail closed.
  • Typed pages use the profile's declared title field. Citation line-list parsing is shared across consumers and accepts whitespace around commas.
  • Invalid concurrency warnings use stderr, preserving the quickstart JSON output channel.

Contributors

Credit includes original proposals, reports and diagnoses, including work completed or extended by maintainers:

Known limitations

  • Windows path fixes are included, but release CI remains Linux-only. Full Windows filesystem-confinement guarantees are not verified; use Linux for untrusted projects until #175 is resolved.

Added

  • OpenAI Codex CLI provider — LLMWIKI_PROVIDER=codex-agent or
    --provider codex-agent delegates chat and structured generation to a locally
    authenticated Codex CLI. The same one-run --provider override is now
    available on compile, refresh, query, watch, rules extract, eval,
    and quickstart. Codex runs non-interactively in an ephemeral read-only
    sandbox with a minimal environment, bounded output and process-tree lifetime,
    interruption cleanup, and independently validated structured results. It
    never reads project .env files or falls back to API-key authentication when
    explicitly selected; embeddings require an explicit existing backend.

  • Skip embeddings on compile — compile({ embeddings: false }) runs page generation, links and the lexical index without any embedding-provider call or pending-embedding retry, for an SDK host that maintains its own semantic index. Omitting it is unchanged.

    The flag closes a gap that is provider-specific. With anthropic or claude-agent a caller can already opt out by not setting VOYAGE_API_KEY, but the openai embedding credential falls back to OPENAI_API_KEY and ollama needs no key at all, so those callers had no way to compile without embedding except to break chat.

    Contributed by @TigerOfCountryYao (#169).

  • Caller system policy on compile — compile({ systemPolicy }) appends deployment-specific editorial or publication guidance to the built-in compile prompts, for SDK hosts that need it without forking the prompts. It is additive rather than a replacement, sits before the source material, and blank or omitted leaves the prompt byte-identical.

    It is advisory rather than enforceable: a policy makes a model more likely to follow a rule, and nothing downstream verifies that it did. Anything that must hold belongs in a lint rule or a trust gate.

    The policy is a prompt modifier, so changing or clearing it regenerates the pages compiled under the previous one, and replaces a pending review candidate produced under it. Each page records the policy's digest, never its text, since the modifier set travels into state.json, page frontmatter and the JSON export. PROMPT_VERSION advances to v2.

    Groundwork by @TigerOfCountryYao (#170), and see #144 for the request this partly answers.

  • Atlas Cloud provider — LLMWIKI_PROVIDER=atlascloud (aliases atlas-cloud, atlas) routes chat and tool calls through the Atlas Cloud gateway, which exposes an OpenAI-compatible API across models from several publishers. Authenticate with ATLASCLOUD_API_KEY or ATLAS_CLOUD_API_KEY; ATLASCLOUD_BASE_URL overrides the endpoint.

    Model ids are namespaced by publisher, and compile extracts concepts through a tool call, so the default is a model Atlas Cloud lists as supporting tools. Embeddings are not wired up: the provider fails closed rather than inheriting OpenAI's semantics, so route them elsewhere with LLMWIKI_EMBEDDING_PROVIDER for semantic search.

    Contributed by @binyangzhu000-sudo (#167).

  • Separate embedding provider — LLMWIKI_EMBEDDING_PROVIDER selects the backend that serves embeddings, independently of LLMWIKI_PROVIDER. This makes split setups possible, such as Claude Agent SDK for generation with a local vLLM instance serving embeddings over its OpenAI-compatible endpoint. Valid values are anthropic, claude-agent, openai, and ollama. minimax and copilot expose no embeddings API, and naming one now fails with a clear error listing the valid values instead of an opaque failure from the provider's embed(). When the variable is set, the provider's own credential is required — VOYAGE_API_KEY for anthropic and claude-agent, OPENAI_API_KEY for openai — unless OPENAI_EMBEDDINGS_BASE_URL points at a self-hosted endpoint, which needs no key. Behaviour is unchanged when the variable is unset.

    Thanks to @knew-inventai for the request (#154).

    Changing the embedding backend invalidates the embedding index. It records the provider, model, and endpoint that produced its vectors, so the next llmwiki compile re-embeds every page. The model name alone is not enough to tell two backends apart — a local server answering to text-embedding-3-small and cloud OpenAI tag a store identically while producing vectors that do not share a space, and nothing downstream would notice. Moving between anthropic and claude-agent does not rebuild: both embed via Voyage with the same model.

    Set OPENAI_EMBEDDINGS_API_KEY to give a separate embeddings endpoint its own credential. Without it the embeddings client reuses OPENAI_API_KEY, which sends your cloud OpenAI key to whatever host OPENAI_EMBEDDINGS_BASE_URL names; llmwiki now warns when that happens, except on localhost. The warning redacts any credential carried in the endpoint URL itself, and the endpoint is hashed rather than stored verbatim in .llmwiki/embeddings.json.

    An index written before llmwiki recorded the endpoint carries only its model name. It is preserved while you run without LLMWIKI_EMBEDDING_PROVIDER or an endpoint override, so upgrading does not re-embed an existing project; under either override the model name cannot establish where the vectors came from, so the next compile rebuilds the index once and records the full configuration from then on.

  • llmwiki rm <source> — Deletes a source and the concept pages derived exclusively from it, leaving pages another source also contributed to untouched. There is no confirmation flag; preview first with --dry-run, which takes no lock and changes nothing. rm refuses cleanly if another llmwiki process holds the project lock, and journals page deletes so a crash mid-removal recovers on the next rm or compile (#60).

    The plan is computed before the lock is taken, so rm recomputes which pages the source owns from fresh state under the lock and refuses the whole removal if anything moved in between — a page became shared, became exclusive, moved to another source, or was added. Nothing is deleted on a refusal; re-running plans against current state. llmwiki watch recompiles on any change under sources/, so this window is an ordinary workflow rather than a rare schedule.

    The source file is deleted before the pages, so a failed page batch leaves the file gone and its state entry behind. Re-running rm with the same name recognizes that pairing as an interrupted removal and finishes the job, rather than reporting "no source matches" while the pages are still on disk. A name occupied by something that is not a valid source — a symlink, a directory — is not treated as an interrupted removal.

    A kept page's body still cites the removed source, which llmwiki lint reports as a broken-citation error. rm warns whenever it keeps a page so the lint failure is not a surprise; it does not edit the citations.

    rm is now a reserved core CLI verb, alongside llmwiki's other top-level commands. A profile that declares a workflow keyed rm now fails validation at load instead of installing; rename the workflow to use that profile.

  • Optional ## Sources section — LLMWIKI_SOURCES_SECTION=off, or --no-sources-section on llmwiki compile, stops page generation from asking the model for a trailing ## Sources section. Unset preserves the prompt byte-for-byte.

    This is for projects that render source attribution themselves. A page already carries its provenance twice — the sources: frontmatter, which the compiler builds from the source files it actually read rather than from anything the model writes, and the inline ^[file.md:1-5] citation markers — so a consumer that displays either one shows the same list a third time in the prose. Nothing downstream reads the section: it is a prompt instruction only, and no linter, exporter, or citation rule parses it.

    Suppressing the request avoids matching a localized heading downstream: under --lang the model may translate ## Sources along with the page. This changes the prompt instruction; it does not enforce the absence of a heading in model output.

    Setting or clearing this preference regenerates affected pages through the existing prompt-modifier fingerprint. Page provenance records sourcesSection=off when disabled. PROMPT_VERSION advances to v3 to identify the implementation with a conditional Sources instruction; the default prompt text is unchanged.

Fixed

  • A shared page kept by llmwiki rm could never be rebuilt — rm records each kept slug in state.frozenSlugs, the same marker compile sets when it notices a deleted source, and that marker was terminal: mergeExtractions skipped a frozen slug outright and nothing removed a slug from the set. The page kept the removed source's prose and its citations permanently, llmwiki lint reported broken-citation at error severity on a page no command could repair, and deleting the page to force a rebuild lost it for good.

    A slug frozen because the current run's extraction failed is still skipped and preserved. A slug carried in persisted state is now a reconciliation marker instead: the page is rebuilt from whatever owners survive, with the removed source's contribution dropped, and a page no live source owns is orphaned rather than held. Fixes #194.

    Contributed by @TigerOfCountryYao (#171).

  • Interlink resolution corrupted prose containing $ sequences — when compile linked a page, it spliced the rewritten body back in with a string replacement, and String.replace interprets $&, $`, $' and $$ there. A page reading The PID is $$ lost a $; one containing a backtick-dollar had its own frontmatter spliced into the middle of the sentence; $& duplicated the body. Shell, sed, awk and Makefile pages carry those sequences as ordinary prose. The rewritten body is now inserted verbatim.

  • Changing the output language left existing pages untouched — llmwiki compile --lang Japanese over an already-compiled project reported "Nothing to compile" and every page kept its previous language, because change detection classified a source by the SHA-256 of its bytes alone and a prompt modifier is not part of the source. The selected modifiers are now recorded in .llmwiki/state.json, and flipping one invalidates the pages it would have changed. Setting LLMWIKI_OUTPUT_LANG has the same effect as the flag.

    Pages also carry a promptModifiers frontmatter entry naming the modifiers active when they were generated, surfaced per page in the JSON export. promptVersion names the prompt implementation and is identical whether or not a modifier was selected, so it could not tell two such pages apart.

    A project that never set a modifier pays nothing on upgrade: its first compile records "none selected" and finds no difference. A project already running with --lang when it upgrades recompiles once, because a state file predating this cannot establish what its pages were generated under.

  • Wikilinks naming an existing page too briefly never resolved — page generation writes a concept's short canonical name while the page it means carries a longer descriptive title, so [[Argo CD]] sat broken next to argo-cd-image-update-ownership-model. Extraction chooses page titles and generation chooses link text, independently and in that order, and nothing reconciled the two. Compile now runs a repair pass after interlink resolution that repoints a link when its slug prefixes exactly one page: [[Argo CD]] becomes [[argo-cd-image-update-ownership-model|Argo CD]].

    Only the link target is rewritten; visible labels, code examples, frontmatter, and literal dollar sequences are preserved. A slug prefixing two live or pending pages is left alone, as is one prefixing none. Pending-only matches wait for approval, which retries repairs. Reads use compile's confined reader, dropping escaping paths and special files before their bytes can enter a write.

    Measured across five compiles of a mixed corpus (two PDFs plus five prose documents), this repaired 21.3% of broken wikilinks on a GPT-5-class model and 16.7% on gpt-4o-mini, with no page's prose changed.

  • Reasoning models could not be used at all — the OpenAI provider hard-coded max_tokens on all three completion paths, and the o-series and GPT-5 families reject it: Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead. The failure came back on the first extraction request, before any page was written, so those models were unusable rather than degraded. The field is now selected from the model id, with LLMWIKI_OPENAI_TOKEN_PARAM to force it for gateways that serve a reasoning model under a private id. The SDK has carried max_tokens as deprecated in favour of max_completion_tokens since 6.x.

    GPT-5.6 models default to reasoning_effort: none for Chat Completions tool compatibility. Older families keep their server default because they do not all accept none. LLMWIKI_OPENAI_REASONING_EFFORT overrides the effort with any of none, minimal, low, medium, high, or xhigh; the chosen model must support that value. These options apply to the shared OpenAI-compatible request paths, including completion, streaming, and tool calls.

    The default is deliberately not applied to the o-series: it accepts a tool-carrying request with the field absent, and rejects some of the values above, so guessing on its behalf would turn a working request into a 400. Set the variable to opt it in.

    An unrecognised value for either variable fails on the first request attempt with the accepted values named, before network access and without retry backoff. Models outside the detected reasoning families retain their previous request shape.

  • Windows: profile path validation rejected every declared directory — on win32, llmwiki template init failed for every template with entity directory must be under 'wiki/', any profile declaring a workflow projectionFile failed to load, and an entity directory declared as wiki/ was wrongly accepted despite containing every reserved subtree — on win32 it was the only entity directory that loaded at all. Declared directories canonicalize to /-joined repo-relative paths, but the containment check built its prefix with the platform separator (\ on Windows), so no nested path ever matched. The lexical profile-path checks now compare POSIX paths directly; native path confinement is unchanged. Reported and diagnosed by @squ1ddy (#163).

  • Windows: broken links in the generated wiki index — the same separator bug on the output side. Entity-page links in wiki/index.md are built from path.relative, which emits \ on win32, so a NESTED entity directory produced the unusable link research\papers/foo.md. Link targets are now normalized to POSIX. Single-level directories were unaffected, which is why this went unnoticed (#163).

  • Windows: native separators in public problem paths — the same separator bug one layer further out, on the reported-problem surface. EntityProblemView.path is documented as project-relative portable content, but both producers returned path.relative output raw, so on win32 llmwiki status, the viewer, context packs, and the JSON export reported wiki\notes\untitled.md where the contract promises wiki/notes/untitled.md. Both now normalize to POSIX. The regression gate was widened to match: instead of naming the two symbols the first fix touched, it now requires every path.relative in the lexical profile layer to be routed through toPosixPath (#163).

  • npm test could not run on Windows at all — vitest's global setup shelled out to npx, which is npx.cmd there and has not been resolvable by child_process without shell: true since the Node 22 hardening for CVE-2024-27980. The setup threw, collection aborted, and vitest reported the unrelated "No test files found". It now invokes the build directly with the running Node binary, needing no shell on any platform.

  • Embedding store dimensions after a full rebuild — when the embedding model changed, the rebuilt store carried the previous vector dimension forward, so switching to a provider whose vectors have a different dimension failed validation on every subsequent compile and never recovered. The rebuilt store now takes its dimension from the newly written vectors.

  • A rebuilt-but-empty embedding index broke every query — a rebuild with nothing eligible to embed persisted a store declaring dimensions: 0, and each later query asserted its query vector against that zero and threw. No compile rewrote the store, so it never recovered. A non-positive stored dimension is now treated as unknown, and a read with no candidates returns before embedding the query at all — which also drops a provider round-trip that could only be scored against an empty pool.

  • OPENAI_EMBEDDINGS_API_KEY was accepted at startup and then ignored — the embeddings client was built only when OPENAI_EMBEDDINGS_BASE_URL was also set, so a configuration supplying just the dedicated key passed validation and then authenticated with the chat client's placeholder credential, failing later as a 401. The dedicated client is now built whenever either the endpoint or the key is configured.

  • A misconfigured LLMWIKI_EMBEDDING_PROVIDER failed late and inconsistently — validation ran inside the embedding call, so the same typo made llmwiki query exit 1, made context retrieval degrade, and made compile warn, retry, and eventually quarantine the affected pages. An unusable name also fell through to the default model and was reported as "the index was built with a different model", which described the wrong problem. The provider guard now checks it at startup, before any work begins, alongside the chat provider's credentials.

Contributors

tienlx91, carmilso, and 11 other contributors
Assets 2
Loading