Skip to content

Releases: crow-hq/llm-wiki-v2

Release list

0.4.0

Choose a tag to compare

@sassarini-marco sassarini-marco released this 12 Oct 19:59
Immutable release. Only release title and notes can be modified.
edf29d1

Added

  • Move a note to another folder: wiki.move_note(path, folder), llmwiki2 move NOTE FOLDER, POST /note/move and a Move button on the note page. Every link to and from the note (See-also entries and Markdown links) is rewritten, the indexes of both folders too, and archived copies get the new archived_from. Edit a folder's description: wiki.describe_folder(path, description), llmwiki2 describe FOLDER "DESCRIPTION", POST /folder/describe and an Edit description button in the sidebar.
  • Long documents with the LLM alone. A source that fits in one call is summarized whole, as before; a longer one is read in parts: librarian/summarize_part takes notes on each block in turn (plain text, the facts first) and librarian/summarize_combine writes the note from them. split_blocks(text, max_chars) in llmw2.agents.state cuts at sections, then paragraphs, then sentences, losing nothing. A document is still one note. The notes of a part over their share are cut at a sentence end, and a part cut by the output limit keeps what it wrote. A warning is logged when the note lacks values (codes, numbers, two-word names) of its part notes. A part that fails fails the step.
  • read_chars (OKF_LLM_READ_CHARS, "Characters per call" in the settings page, kept per provider): the characters one request may hold. Empty means automatic: 100000 on OpenRouter, OpenAI and Gemini, 16000 on Ollama and custom servers (Provider.read_chars, WikiConfig.effective_read_chars).
  • note_chars (OKF_NOTE_CHARS, "Note length" in the settings page): about how long a note body may be. Empty means 10000, and at most half of what one call holds. It is asked of the model, never enforced: a body over 1.2 times the length asked is logged, not cut.
  • note_length (OKF_NOTE_LENGTH): proportional or fixed. A proportional note is about a quarter of its source (NOTE_RATIO), never below note_chars, up to what a call holds and NOTE_MAX (45000); a merge asks for the existing note plus that share (WikiConfig.target_note_chars). fixed is a note of about note_chars.
  • revisions (OKF_REVISIONS): replace or merge. A source that revises the one document a note cites replaces the note (a summary of the new version, a "Changes from the previous version" section with each changed value and "previously ...", and a "From the previous version" section with the sentences that still hold), and the old body goes to a hidden .archive/ (WikiStore.archive_note, previous_version in the frontmatter). When a question asks about an earlier version, the researcher reads the archived copy and answers with the earlier value. When the guard does not pass (not the same document, or a body not written by code alone), the source is merged as usual. The guard is recorded as a decision whose decider is the new value "code".
  • The web page of a note revised in place has a Versions block that links its archived version; an archived copy is labelled as such, links the current note and the version before it, and has no delete button.
  • Every note records generated.body_hash, the hash of its main body, on create and on every update.
  • votes (OKF_VOTES, default 1) and vote_band (OKF_VOTE_BAND, default 0.05) for CROW. With votes above 1, a routing decision close to tau_route, to a tie between two folders or to tau_path is asked votes times and the classifier's answers are averaged, so its noise flips the decision less often between runs. The route_step trace ends with (n votes) when it was averaged.
  • fallback_menu (OKF_FALLBACK_MENU, visited or tree, default visited) for CROW, an extension of the paper. With tree, the routing fallback shows the LLM the folders the classifier visited and then the other folders of the wiki (at most 8000 characters; the visited ones are never left out), so it can pick a folder the classifier never reached. It is in the settings page next to votes.
  • A deterministic state for routing. CROW route reads a digest built from the source text (the text whole if short; else its summary or introduction, outline and excerpts) instead of the LLM-written summary, which changes on every run. Candidate.state holds it and a custom summarize may fill it; StepContext.route_state(cand) returns it. For a source longer than twice note_chars the digest leaves out the source's own summary. StepContext.candidate is the note summarize wrote, set before route.
  • An output cap on every model call made through StepContext.ask_json and ask_text, after a model ran away to 131072 tokens. Both take reply_chars (default note_chars; decisions use 2000), and LLM.chat, json and complete take max_tokens. With keep_cut, ask_text and LLM.complete return a reply cut by the cap as written, with a warning, and append it to cut when given. LLM.chat takes json_mode, which asks for response_format json_object. reply_tokens(chars) is 3 times the length at 3.5 characters a token, at least 1024, plus 32000 when the model may think. The field is max_completion_tokens on OpenAI (Provider.max_tokens_field); a provider that refuses a cap is asked again without it, and an extra_body cap wins.
  • llm.seed (OKF_LLM_SEED) and llm.pin_provider (OKF_LLM_PIN_PROVIDER, comma-separated), both on the settings page under "Model parameters". The seed is sent with every request. On OpenRouter the pinned providers answer in order, with no fallback to others and only if they support every parameter sent; other providers ignore the pin with a warning. An extra_body["provider"] wins. Neither is set by default.
  • Every line of the usage log records the provider that served the call (when the response says), the temperature, the seed and the finish_reason.
  • Prompts.fingerprint() and decision_hash(steps, cfg, prompts) in llmw2.agents.steps.
  • The settings page warns, when testing an Ollama model, if its num_ctx is smaller than a call can need.
  • A stable id in the frontmatter of every new note (12 hex characters). Updates, moves and archived copies keep it; older notes get none; it is never used in links or indexes. Wiki.find_note(id); check reports two notes with the same id.
  • Routing audit (CROW): wiki.audit_routes(min_confidence=0.7, apply=False), llmwiki2 audit [--min-confidence X] [--apply] and POST /audit. Routes each note of the tree again with route's classifier decisions and never its LLM fallback, and proposes the notes whose folder differs, not deeper in the note's own branch, with a confidence of at least min_confidence (default AUDIT_MIN); apply moves them. No LLM call, never automatic. It returns RouteProposals; taxonomy_health below returns FolderHealths. All three are exported from llmw2.
  • Taxonomy health: the CROW ingest log line ends , routed by fallback when the LLM fallback chose the folder, and wiki.taxonomy_health(min_filed=5, max_share=0.3), llmwiki2 health and GET /taxonomy read log.md (no model) for the new subfolders and fallbacks per folder and month, flagging the folders with too many.
  • Benchmarks in benchmarks/ (stability of the filing decisions, filing errors judged blind by readers, facts that survive into long notes, replay of model answers), the corpora they use and the stored judgements, with a README.

Changed

  • Behaviour change: the confidence of a classifier Choice is computed by the library from the probabilities, on the scale the CROW thresholds are calibrated on (llmw2.models.classifier.confidence), and the server's confidence field is ignored. Servers disagree on what that field means (Ollama and Laya send one measure, some OpenRouter providers another), so the same answer took different decisions on different servers; with a classifier on Ollama, CROW never merged an update. One set of thresholds now works on every server. On Jev nothing changes but a decision exactly at a threshold: 27 of 17,558 stored answers, none a merge. A server answer without confidence is no longer malformed.
  • Behaviour change: Laya's preset tau_cons is 0.26 instead of 0.05: the same merges on the new scale. A custom classifier's tau_route (0.6) is now read on Jev's scale too; if you set OKF_TAU_CONS or OKF_TAU_ROUTE for a custom server, set them again on it.
  • Behaviour change: revisions defaults to replace: a source that revises the document a note cites replaces the note and archives the old body. OKF_REVISIONS=merge keeps 0.3.0's merge.
  • Behaviour change: note_length defaults to proportional: a note grows with its source, to about a quarter of it. OKF_NOTE_LENGTH=fixed keeps 0.3.0's length.
  • Behaviour change: every note now records generated.body_hash.
  • Behaviour change: the decision hash changes: runs before this one hash differently. It now also covers read_chars, note_chars, note_length (unless fixed), relate_rare_values, answer_conflicts and strict_new_folders (only when false), the size and version of the routing state, the fallback menu's character limit, SUMMARY_VERSION, the seed and the pinned providers, and the text of note_rules and merge changed. The ingest log line ends with (name hash, model) instead of (name hash), plus , via providers when providers are pinned.
  • Behaviour change: a reply cut by the output limit is asked once more, shorter; if it is cut again, the call raises ReplyCut (a ModelError) and the source fails, where before a long note was filed with a warning (or ran away). Direct ctx.llm calls stay uncapped, and the cap is not in the decision hash.
  • summarize reads a source that fits in one call without a length rule, and its output cap follows the source length. The length rule of shared/note_rules is the placeholder {{length_rule}}, left empty by librarian/summarize and librarian/merge_source.
  • CROW route and consolidate read the source, not the...
Read more