Repository navigation
Lorebase 0.3.0
Knowledge that stays correct: a reconciliation queue for contradictions between Sources, corrections that survive Source syncs, document versions, ingestion modes and stewards, Traits and lenses in search and context packs, the merge of text values, and the resolution of words to schema and values with consolidation from use and coverage signals; the web app gains a Reconciliation screen and a passage-by-passage Conflict resolver. Pre-1.0: this minor version changes interfaces; read the breaking changes before upgrading.
Ten migrations (0038 to 0047). New configuration sections keeper.reconciliation, keeper.stewards, scribe.reconciliation, scribe.consolidation, store.values, store.resolve_misses, store.coverage, search.resolve and read.coverage, and new keys store.resolve_misses.caller_key, store.storage.hnsw_m, store.storage.hnsw_ef_construction, search.semantic.iterative_scan and search.semantic.max_scan_tuples. New endpoints POST /domains/{d}/resolve and /domains/{d}/reconciliation; responses add coverage to POST /query and POST /resolve, semantic_truncated to POST /search, can.settings to GET /domains/{d}, and source_key, version, replaces and status to documents. A Source sync no longer overwrites a value someone corrected on the target, and automatic merges into a protected branch wait for steward and role approvals (below, Breaking changes). The lines link the documentation of each change: the work landed directly on main, without pull requests to link.
Breaking changes
- Source syncs no longer put back a value someone corrected on the target: an
auto_when_cleanSource that merged silently stops at an open Proposal with aDV1orDV2Conflict when upstream changes such a value, and on its first run after migration0040. Setlocal_edits: source_winson the Source to keep the old behavior (below, Changed and Upgrade notes). - Source syncs with
auto_when_cleanorauto_with_authorityand extraction indirectmode no longer merge into a protected branch whose rules includerequired_approvals_from_stewardsorrequired_roles; the Proposal stays open (merge_blocked: approvals,direct_blocked: approvals). Installations without those rules see no change (below, Changed). - A
textAttribute edited in different sentences on two Branches merges automatically (resolverpolicy:text_merge) instead of raising a Conflict. Settext_merge: falsein the Domain'smerge_policysetting to keep the Conflict (below, Changed). - Semantic search reads the HNSW index at any size: unfiltered and broad-filter lists are approximate where they were exact (recall 0.92 to 0.99 at 300,000 vectors); a one-Kind filter stays exact (below, Changed).
Added
- Resolution of words to schema and values (design 8.9, AC-39).
POST /domains/{d}/resolve {text, kinds?, levels?, query_vector?, limit?, branch?, at?}returns ranked candidates the words of a text may stand for: Kinds, Attributes (by name or alias, with their type) and stored values with a ready-to-usewherepredicate and the count of Records holding them, each with its level (exact,synonym,trigram,embedding), score and the words matched, plus the words nothing matched. It never interprets the question, runs a query or calls a model; the embedding level only uses a caller'squery_vectorwhen no other level matched. Candidates never name a Kind or Attribute the caller may not read. Heraldlorebase_resolve, CLIlorebase resolve TEXT, Python clientrecords.resolve(sync and async). Settingssearch.resolve(agents). - Value index:
mat_valuesholds per branch the distinct short values of selected Attributes, kept by triggers onmat_recordsin the writing transaction (migration0045, additive). Enums are selected always, plain strings while their distinct values stay few (store.values); the Scribevalue_indexpass (scribe.scheduler.value_index_interval_s, default 300 s) andlorebase-scribe value-index DOMAIN [--rebuild]decide and backfill the selection. Sensitive and vault values are never indexed. A pass that changes the selection fences only its own Domain's writers (a transaction advisory lock taken shared by the triggers), for at moststore.values.pass_timeout_ms; a schema change that makes an Attribute ineligible removes its values in the same commit. At 100,000 Records resolution answers in 11.2 ms at p95 (levels 1-2); the triggers cost about 90 µs per statement where nothing is held and 0.3 ms per updated Record whose held values change (performance, operations). - Value index (design 8.9): an Attribute's distinct-value limit now holds between
value_indexpasses. The limit ismax_values:hard_max_distinctfor enums andresolvable: true,max_distinct×hysteresisfor plain strings. The write that takes a branch past it marks the Attributeoverflowedin the same transaction: resolution stops returning its values at once, later writes stop indexing it, and the next pass drops it. A schema commit or merge on the default branch applies a new limit in the same commit. Reads also skip a branch past its limit that is not yet marked. - Migration
0047addsmat_value_sizes, the per-branch count of held values kept by the value triggers, andmax_valuesandoverflowedtomat_value_attributes. Passes read the sizes instead of countingmat_values. On upgrade, existing selections gethard_max_distinctand their sizes are counted; the first pass of each Domain counts its sizes again and sets its own limits. Trigger cost: about 90 µs per statement where nothing is held, as before; 0.3 to 0.4 ms per updated Record whose held values change, against 0.29 ms before (measured withtests/perf/resolve_bench.pyon a loaded machine; performance). - Schema:
aliaseson Kinds and Attributes,value_aliases(stored value to other words) andresolvable: true|falseon Attributes; absent by default, so existing schemas keep their hashes. Validation refusesresolvableandvalue_aliaseson types other than string and enum (alone or in arrays), on sensitive and on link Attributes, and value aliases of an enum that name no value (schema.attribute_flag_invalid). The web Modeler keeps them on round trips. - Herald
lorebase_querynow documents counting with"aggregate": {"group_by": attr}and points tolorebase_resolve. - Consolidation from use (design 8.10, AC-40): with
store.resolve_misses.enabled(off by default), Keeper keeps the phrasesPOST /resolvematched to nothing per Domain, branch and phrase (counts, first and last seen, digests of distinct callers; migration0046), off the request path and only for head reads of callers holdingsearch.resolve.misses.permissionDomain-wide; requests opt out withrecord_misses: false, and searches narrowed bykindsorlevelsare not recorded. One caller adds and holds a bounded number of phrases; a full Domain evicts its least missed phrase. Phrases expire afterretention_days(30) and go with their Domain, and so do the vocabulary suggestions quoting them (their share removed, a suggestion left without phrases deleted, decided or not). A Scribevocabulary_consolidationpass (scribe.scheduler.consolidation_interval_s, CLIlorebase-scribe consolidate DOMAIN) turns frequent misses into schema suggestions of the new typesvalue_alias,attribute_alias,kind_alias(near values and names; never for sensitive, vault, derived or non-resolvable Attributes, never a phrase that already names something) and, withscribe.consolidation.llm.enabled,attribute_description(taskschema_consolidation; the model sees names and phrases only). They are reviewed like other suggestions; accepting one opens a schema change Proposal. Vocabulary suggestions have their own cap (schema.suggestions.max_vocabulary_tracked) so they cannot crowd out extraction's; they are withheld from reviewers who may not read their Kind's Records or Attribute's values; an alias that would name two things is refused at acceptance.lorebase-scribe misses purge DOMAINdeletes a Domain's misses and vocabulary suggestions. - Coverage signals (design 8.11, AC-41):
POST /queryandPOST /resolveanswer with acoverageblock (coverage: falseleaves it out; defaultread.coverage.enabled): the share of Records missing each Attribute used, Source freshness (last successful sync,stale_afterexceeded, Records unseen), the phrases unknown in the Domain, and one note per signal. It reads per-Kind statistics of the default branch head (mat_coverage, migration0046) counted by a Scribecoveragepass (scribe.scheduler.coverage_interval_s, CLIlorebase-scribe coverage DOMAIN) from one snapshot per Kind, Records unseen by Sources recounted every pass; the last successful run of a Source is one probe of the partial indexix_source_runs_last_succeeded; masked Attributes and Kinds outside the caller's scope are never described, and other branches and Revisions get no statistics. The two lookups take p95 3 ms at 100,000 Records and 600,000 Source runs. - Python client
CoverageonQueryResultandResolveResult,coverageonQueryRequestandResolveRequest,record_missesonResolveRequest, the vocabulary suggestion types andattribute,value,descriptiononSchemaSuggestion; CLIlorebase queryandlorebase resolveprint the coverage notes; Heraldlorebase_queryandlorebase_resolveend withcoverage:lines. - Web app, Review > Schema suggestions: vocabulary suggestions with the alias, the Attribute, the value they stand for or the proposed description, and the missed phrases as evidence; their Documents cell stays empty, as they come from no document.
store.storage.hnsw_m(default null: pgvector's 16) andstore.storage.hnsw_ef_construction(default 128; pgvector's own default is 64) set the build parameters of the HNSW index of embeddings, for a new installation (migration0001) and for indexesstorage applybuilds later.storage applydoes not rebuild an existing index: installations from before this release keep ef_construction 64 until they rebuild it withALTER INDEX ... SETandREINDEX INDEX CONCURRENTLY(operations). Measured at 300,000 vectors, ef_construction 128 raises mean recall@20 from 0.910 to 0.988 unfiltered and from 0.964 to 0.989 with a 40% filter, same index size, build 66 s instead of 50 s. Documentation now says the semantic list is approximate and gives these figures (search).- Incremental Sources and upstream deletions (design 9.1). Each run of an incremental Source records the Records it read, changed or not (
source_written.last_seen_runandlast_seen_at, migration0043, additive). The new Source settingstale_after(ISO 8601 duration, default null: off; incremental Sources only) makes the reconciliation sweep raise astalecandidate with signalsource_unseenfor each Record the Source owns that no run has read for longer; nothing is deleted. Such a candidate takeskeep_currentor a dismissal (another choice answers 422) and goesobsoleteonce a run sees the Record again (source_seen).scribe.reconciliation.stale.source_unseen(default true, Domain override allowed) turns the check off (Scribe). - Web app, Modeler: the Source editor sets
stale_after; the Reconciliation screen explains thesource_unseensignal. - Document versions and lifecycle (design 9.2, AC-35). An upload may name a
source_key(multipart field or JSON member; at mostkeeper.documents.max_source_key_chars, 1024): the bytes of the key's active version return it (200), other bytes become version n + 1 withreplacesthe previous one, nowsuperseded, auditeddocument.version. Keyless uploads keep their content addressing.GET .../documents?source_key=andGET .../documents/{id}/versionslist versions; documents showsource_key,version,replacesandstatus(active,superseded,retired,purged), and a document's detailsopen_stale_candidates,can_retireandcan_purge. Migration0041, metadata only (operations). - The
document_version_checkjob: for a new version, each value whose current provenance is extraction from an earlier version, and that the new version does not propose again, has its old evidence quote looked for verbatim (white space aside) in the new text; a gone or purged quote raises astalereconciliation candidate with signalevidence_gone({document, from_version, to_version, quote}), masked like evidence for sensitive Attributes. Nothing is retracted and no model is called. Bounded byscribe.extraction.versions.max_values_per_job(5000) with continuations; a Domain turns it off withreconciliation.triggers.document_versions: false(extraction). - Retirement and purge of documents:
POST .../documents/{id}/retirement {reason}(new permissiondocument.retire, held bymaintainerandadmin) runs the check with empty text and stops extraction for good (409document-retiredwhen an upload asks to extract it; extraction jobs naming it fail).POST .../documents/{id}/purge {reason, confirm}(new permissiondocument.purge,adminonly;confirmis the document id) deletes the bytes, parsed text, chunks and cached answers of the document and replaces its evidence quotes in Proposal items, candidates and schema suggestions by[erased], keeping provenance rows; irreversible, auditeddocument.purgein the same transaction (operations). - Python client
client.documents.versions|iterate_versions|retire|purgeandsource_keyonupload,upload_urlandlist; CLIlorebase documents upload FILE --key KEY,lorebase documents list --key KEY,lorebase documents versions ID,lorebase documents retire ID --reason TEXT,lorebase documents purge ID --reason TEXT --confirm ID. - Web app, Documents: key, version and status in the library and on the document page, the versions of a document, its open
stalecandidates linked to the Reconciliation queue, a source key on upload, and Retire and Purge (typing the document id to confirm) for those who hold the permissions. - Read presets (design 8.8).
POST /searchand thesearchandqueryseeds ofPOST /contexttaketraits: a Trait stands for the Kinds implementing it that the caller may read, andkindswithtraitssearch the union. Herald'slorebase_searchandlorebase_contexttaketraitsandlens, the CLI haslorebase search --traitand--lensandlorebase context --traitand--lens. - Lenses:
lenses.{name}in the schema document is a named read preset (description,kinds,traits,relations,depth,include_schema,include_recent_changes,include_lineage,budget_tokens), a schema element with the id prefixlns_that merges and is validated like the others (schema.lens_reference_unknownfor a name the schema does not hold,schema.lens_invalidfor a depth below 0 or a budget below 1).lensonPOST /searchandPOST /contextsupplies the defaults: members of the request win, a lens over the caps is served at them (budget.capped), and the lens's Kinds and Traits limit neighbors andsearchandqueryseeds whilerefsseeds are always kept. A lens is not a permission: Kinds, Traits and Relations the caller may not read are dropped from it, and an unknown lens, or one whose Kinds are all unreadable, is 422lens_unknown; withkeeper.auth.authz.invisible_kinds, a Trait none of whose implementing Kinds is readable is 422unknown_traitlike an unknown one.GET /schemalists the lenses; Herald adds the resourcelorebase://{domain}/lenses/{lens}and aLENSESpart tolorebase_schema(agents). - Vector search that keeps its limit under filters: with pgvector 0.8 or later and
search.semantic.iterative_scan: auto(default) the semantic list runs underhnsw.iterative_scan = relaxed_orderbounded bysearch.semantic.max_scan_tuples(20000) and is ordered again by exact distance; otherwise the index returnssearch.candidates * search.semantic.overfetch_factor(4) nearest rows before the filters. The response sayssemantic_truncatedwhen the scan bound left a list short while more Records match./readyzreports the case in avector_searchcheck (search). - Python client:
traitsandlensonSearchRequestandContextRequest,semantic_truncatedonSearchResult, and thelensessection ofschema.element. - Reconciliation queue (design 10.6): values on which Sources disagree no longer pass unnoticed when a one-sided write replaces another Source's value. Scribe raises candidates of type
contradiction,superseded,staleandduplicatewhile extraction items are assembled, before a Source sync merges, and in thereconcile_sweepjob (scribe.reconciliation.sweep_interval_s, off by default); nothing is changed, hidden or decided because a candidate exists. Stages run cheapest first: identity, values, normalization (units of a shipped table, numeric tolerance, case and punctuation, negation words) and an optional semantic stage (off by default) that marks paraphrasesequivalentand compares stored Record vectors for duplicates. Candidates are fingerprinted by their values, so a dismissal holds until a new value appears (Scribe). - Reconciliation decisions in Keeper under
/api/v1/domains/{d}/reconciliation: list with filters, get,POST .../{id}/resolution(take a claim, keep the current value, a custom value, or a policyauthority,newest_wins,prefer_current,prefer_incoming;dry_run),POST .../{id}/dismissal,POST .../{id}/reopen,POST .../reconciliation:resolvein bulk (at mostkeeper.reconciliation.bulk_max) andPOST .../reconciliation/sweeps(202). Values are written by the Kind'supdate_{kind}with areconciliationprovenance; when the branch needs review the write goes on a newreconcile/{candidate}-{random}branch and Proposal. Duplicates merge into a surviving Record through a Proposal. A candidate whose decision waits in an open Proposal refuses new decisions (409), and a claim or policy decision is refused when a newer value was written since detection. Decisions needconflict.resolveplus what the write needs, and clearance for sensitive or vault values, and are audited (reconciliation.resolve,reconciliation.dismiss,reconciliation.reopen). - Proposal check
reconciliation:warn(default) lists the open candidates of a Proposal,blockstops the merge while contradictions are open (409reconciliation-open). Set bykeeper.reconciliation.merge_check,scribe.reconciliation.merge_checkfor the merges Scribe runs, and the Domain settingreconciliation_check; the stricter one applies. - Valid time: an Attribute may declare
valid_time: true(not on derived Attributes). Its values carry a validity period on their provenance, never in content: from Source mappings (attribute_mapping.{attribute}.valid_from,valid_to), from tier 2 extraction when the dates are in the quote, and from thevalidparameter ofupdate_{kind}. Disjoint periods make asupersededcandidate instead of a contradiction. - Python client
client.reconciliation(sync and async) and CLIlorebase reconcile list|show|resolve|dismiss|reopen. - Web app: a Reconciliation screen (
/{domain}/reconcile) with a tab and count per type, filters by status, Kind, Attribute, source and signal, the claims of a candidate side by side with source, time, period and evidence, decisions with a preview and a required reason, duplicate merges, bulk decisions and sweeps. The Record page marks values with open candidates, the Proposal page links the candidates of its check, and the Source editor names the validity columns ofvalid_timeAttributes. New settingsVITE_RECONCILIATION_PAGE_SIZEandVITE_RECONCILIATION_COUNT_LIMIT(tour). - Metrics
lorebase.scribe.reconciliation.candidates,lorebase.scribe.reconciliation.sweep.durationandlorebase.scribe.reconciliation.open(metrics). - Ingestion modes of extraction (design 9.2, AC-36): a Domain's
settings.extractionchoosesreview_all(nothing accepted at once),doubts_only(the default, unchanged behaviour) ordirect, withauto_accept_threshold,min_confidenceandrulesbysource_keyprefix (longest wins). Tightening always applies; loosening only for the members inscribe.extraction.domain_may_loosen(default[mode]), others are ignored and listed inignored_settingsofGET /domains/{d}.PATCH /domains/{d}takessettings.extraction(domain.settings, auditeddomain.updatewith before and after values). New settingsscribe.extraction.mode(directrefused at startup),domain_may_loosen,direct_merge_protected(extraction). directingestion: Scribe merges an extraction Proposal itself only when every item came from tier 0 and none is pending, the target accepts it and the merge is clean and not blocked by contradictions; any tier 1 or 2 item caps the job atdoubts_only(mode_capped: tier). The job result saysmode,direct_blocked(tier,pending_items,protected,reconciliation,conflicts),direct_mergedandmerged_revision; the merge is auditedproposal.mergewithmode: direct.- Stewards (design 10.7, AC-37):
stewards: [role, ...]ondomainand per Kind in the schema (an undefined role is the warningschema.steward_role_unknown, returned inwarningsbyschema:validate). Proposals and reconciliation candidates carrysteward_rolesand, for callers withprincipal.read, computedassignees(keeper.stewards.max_assignees_listed, 50);GET .../proposalsandGET .../reconciliationtakeassigned=me|any|none. A principal who cannot read a Kind the work touches is never an assignee. Proposals record the Kinds they touch (proposals.kinds, migration0042). - Protection rule
required_approvals_from_stewards(keeper.proposals, Domain setting of the same name; default 0): approvals that must come from assignees, the Proposal needing the larger of it andrequired_approvals; too few assignees answers 409stewards-unavailable. The approval block showsrequired_from_stewards,steward_approvalsandstewards_unavailable. - Protection rule
required_roles(keeper.proposals, Domain setting of the same name; default empty) (design 10.3): each listed RBAC role must have approved a Proposal into a protected branch, through a principal holding it who may review the Proposal. A merge missing one answers 409approvals-requirednaming the roles; Proposal approvals showrequired_rolesandmissing_roles. - Python client
assignedonproposals.list|iterateandreconciliation.list|iterate, steward fields on Proposals and candidates,settingsondomains.update; CLIlorebase proposal list --mineandlorebase reconcile list --mine. Web app: an "Assigned to me" filter and the steward roles and assignees on Review and Reconciliation, the steward approvals on the Proposal page, and the ingestion mode and ignored settings on the Domain settings screen. - A purge no longer races a running extraction job of the same document: the purge and every write of the job (text, chunks, items, candidates, suggestions) lock the document row, the job stops (
cancelled) when the document was retired or purged meanwhile, the purge cancels the queued and running extraction jobs of the document (jobs_cancelledin its audit entry), and a model answer is cached only while its chunk is stored. GET /domains/{d}andPATCH /domains/{d}reportcan.settings: whether the caller holdsdomain.settings, computed by the authorizer (null in the Domain list).- Web app, Settings: the ingestion settings of a Domain (mode, auto-accept threshold, minimum confidence, rules by source key prefix) are editable for callers with
domain.settings, with Keeper's validation errors at the fields they name and ignored values marked; others see the settings read only. - Web app: the Explorer's Kind filter offers the Traits of the schema, and a lens selector (Explorer search, and the Retrieval lab's searches and context pack) appears when the schema declares lenses.
Changed
- Automatic merges no longer skip the rules that name who must approve (design 9.1). A Source sync with
auto_when_cleanorauto_with_authorityand an extraction job indirectmode merged into a protected branch, withscribe.sources.auto_merge_protectedandscribe.extraction.direct_merge_protectedon (the default), whatever its approval rules said. Those settings still waive the plainrequired_approvalscount, but when the target's effective rules include steward approvals (required_approvals_from_stewardsabove 0) or required roles (a non-emptyrequired_roles), fromkeeper.proposalsor the Domain setting of the same name, the merge is not performed and the Proposal stays open: the Source run's stats saymerge_blocked: approvals, the extraction job's resultdirect_blocked: approvals, both withblocking_rules. The rules are evaluated by one implementation that Keeper and Scribe share (lorebase_store.approval_rules); Scribe readskeeper.proposalsfrom the shared configuration. Installations without steward or role rules see no change (operations). - Semantic search reads the HNSW index at any size (design 8.8). The nearest vectors of the searched Kinds are taken from
embeddingsalone and joined to the Records, instead of joining first (which sorted every vector of the model): 25 ms against 329 ms p50 unfiltered at 300,000 vectors, 34.5 ms p95 against 513, inside the 200 ms budget. Unfiltered and broad-filter lists are approximate (HNSW recall 0.92 to 0.99 here) where they were exact; a one-Kind filter is exact. Over-fetch (iterative_scan: off) restricts to the Kinds' vectors before its limit. - Web app: a dialog opens with focus on its first control (a confirmation on its cancel button), or on its title when it has none, instead of on its Close button.
- The
include_schema,include_recent_changesandinclude_lineagemembers of a context request are unset by default (a lens may set them) and behave astruewithout one; the Python client'sContextRequestno longer sends them unless given. - Object type code 10 (
schema.lens) joins the stored object types; no migration is needed. - Source syncs: corrections stick (design 9.1, AC-34). A Source now writes only the values that changed upstream since it last wrote them, kept per Source as hashes in
source_written(migration0040, advanced in the transaction that merges the Source's Proposal). A person's correction on the target is no longer put back by the next sync while upstream did not change; when upstream does change that value, the Source's Proposal holds aDV1Conflict whose base is the Source's previous value, and a snapshot run that would delete a Record holding someone else's values holds aDV2, so anauto_when_cleanSource waits for a person in those cases. The new Source settinglocal_editschooses:review(default),keep(resolved at once for the person's value, resolverpolicy:local_edits) orsource_wins. To get the old behavior, setlocal_edits: source_winson the Source. Underauto_with_authority, Attributes whoseauthorityis the Source are overwritten as before. Links a person removed or added are left until upstream changes them. A Record kept by aDV2keeps its Links (the merge restores Links the deleting side removed with it), and a Record someone else deleted that comes back from upstream comes back with its Links. Underkeep, each run also decides again the Conflicts a person reopened by editing a kept value while the Proposal was open. Each local edit is audited assource.local_edit_conflict, and run stats countlocal_editsandseeded. The first run of each existing Source seeds the state; a value it did not write that differs from upstream counts as a local edit that once (operations, Scribe). - Extraction: a value item over a value a person set last (an Operation, or a resolution or reconciliation decision by a principal) stays pending whatever its confidence, with hold reason
local_edit(held_local_editin the job stats) (extraction). - Web app, Modeler: the Source editor sets
local_edits. - Merge: a
textAttribute edited in different places on two Branches now merges automatically instead of being a Conflict. The values are compared by sentence (or paragraph) against the base; edits to different sentences are both kept, the result is recorded as an automatic resolution with resolverpolicy:text_merge, and the Review screen lists it under "Resolved by policy". Edits to the same sentence, or two insertions at the same place, are still a Conflict (neighbouring sentences rewritten on each side merge), which now carriesdetails.text_hunks(the whole text in order, with the merged clean parts and the conflicting ones); its fingerprint is unchanged. Other types (string,uri, enums, numbers,json, lists) and values added on both sides are not affected. To get the old behavior settext_mergetofalsein the Domain'smerge_policysetting, or only for a Kind, Relation or Attribute withtext_merge_elementsortext_merge_attributes;text_merge_granularity(sentenceorparagraph) andtext_merge_max_chars(characters over the three values, default 200000; fixed caps on the number of sentences and on the cost of repetitive text also apply) tune it (design 6.8, 6.10). - Web app, Conflict resolver: a text value that changed in the same sentences on both Branches (a DV1 with
text_hunks) opens as passages: unchanged text folded away, each conflicting passage with Ours, Theirs and Base and a choice (ours, theirs, both in either order, base, or edit in place), a live preview with its length against the Attribute's limits, and "Use merged text" once every passage has a choice.[and]move between passages and1to4and0choose. - Web app, Conflict resolver: string values that moved from the base are shown word by word (added and removed words marked, with text for screen readers), and the Ours and Theirs columns carry the names of the target and source Branches. In the custom editor, "Copy base", "Copy ours" and "Copy theirs" load a side into the field. Saving a resolution opens the next open Conflict. Outside text fields
o,tandbpick a side,eopens the custom editor,sor Enter saves andnmoves to the next open Conflict. Leaving a Conflict with unsaved changes asks first, and "Apply resolutions" asks when Conflicts are still open or the one on screen is unsaved. The bulk choice uses the same names as the resolver ("Take ours", "Restore the Record") instead of the raw choice. - The release workflow can be run by hand (
workflow_dispatch, inputtag) on an existing tag, and publishes the ten packages to PyPI through trusted publishing, each from its own environment (pypi-<package>), when the repository variablePYPI_TRUSTED_PUBLISHINGistrue(release process). - Documentation: the release verification commands name 0.2.3 and the identity of a run started by hand; the README, the quickstart and the MCP guide say how to install
lorebase-cliandlorebase-heraldfrom PyPI.
Fixed
- Web app, Modeler: a save also dropped members the editor does not model inside nested objects: a Kind's
search, the Trait implementations of a Kind, Attributeconstraints, Operation parameters and preconditions, and a Source'srelation_mappingentries andtarget_branch_policy. They are now kept as they were. - Semantic search under iterative scans (pgvector 0.8 or later, the default) left
hnsw.ef_searchat pgvector's 40, so each pass of the index scan offered 40 candidates and recall@20 was about 0.92 at 300,000 vectors. The read transaction now raises it tosearch.candidates * search.semantic.overfetch_factor(800 by default, capped at 1000) with and without iterative scans: at 300,000 vectors the 40% filter goes from 0.929 to 0.964, p95 61 ms. The unfiltered list stays at 0.910 with the default index (a query in a region the graph does not reach); see the new HNSW build settings (Added) (performance). docs/reference/configuration.mddid not follow the changed description ofsearch.semantic.overfetch_factor: the page is generated from the comments ofconfig/lorebase.example.yaml, which now carries it.- Web app, Modeler: in live mode, saving the schema dropped the lenses, the
stewardsof the Domain and of Kinds, and any member or section of the document the editor does not model. The editor now carries them through untouched and writes them back as they were. - Semantic search without iterative scans (pgvector before 0.8, or
search.semantic.iterative_scan: off) over-fetched less than configured: an HNSW scan ends afterhnsw.ef_searchrows (40), sosearch.candidates * search.semantic.overfetch_factornearest rows never came back, and a filter keeping 1% of the Records left about 0.2 results of 20. The read transaction now raiseshnsw.ef_searchto the over-fetch count, up to pgvector's limit of 1000: 7.8 results of 20 on 100,000 vectors, flaggedsemantic_truncated(performance). - Web app, mock mode: accepting an attribute schema suggestion reports an update of its Kind, as Keeper does.
Security
- Resolution misses: caller digests are HMAC-SHA256 under
store.resolve_misses.caller_key, required whenstore.resolve_misses.enabledis on (at least 32 characters, asecret://reference), so a reader ofresolve_missescan no longer confirm a guessed principal id. Changing the key restarts the count of distinct callers; a caller seen under both keys counts twice until its phrases expire, andlorebase-scribe misses purge DOMAINstarts clean. Migration0047clears the digests stored before it.
Not done yet
- Corrections that stick: the last-written state of a Source is not shown in the web app or the API.
- Reconciliation: no queries by valid time (periods classify claims, reads stay at Revisions), no key aliases after a duplicate merge (references to the absorbed key are not redirected), and no MCP tool for the queue. In the web app, tab counts stop at
VITE_RECONCILIATION_COUNT_LIMITand the signal filter and ranking apply to the pages loaded. - Allowed merge origins per protected branch (roadmap).
- Resolution misses are not linked to the next query of the same agent session, and context packs and search carry no coverage block (roadmap).
- The performance targets of the design at 10M Records on the reference hardware, and the 1M-Record neighborhood on branches other than
main, just over budget when the working set does not fit in memory (performance). - The stability guarantees for
/api/v1bind from 1.0 (API stability).
Upgrade notes
- Where steward approvals or required roles apply to a protected branch, Source and direct-extraction merges into it now wait for review (above, Changed); give Scribe the same configuration file as Keeper so it sees
keeper.proposals. Existing embedding indexes keep ef_construction 64 until rebuilt (operations). stale_aftersuits incremental feeds that deliver their live rows again within the window; a feed that sends only changed rows flags every unchanged Record oncestale_afterpasses (false positives): leave it null there (Scribe).- Migration
0044buildsix_mat_records_object_hashonmat_records (domain_id, branch, object_hash)per partition withCREATE INDEX CONCURRENTLY(writes continue; no restart; extra space about 40 bytes per row). See operations. - Migration
0038createsreconciliation_candidatesand adds two nullable columns toprovenance(no backfill);0039adds indexes to the candidates. Run the migrations to0039, then restart Keeper and Scribe. Detection is on by default (scribe.reconciliation.enabled) and only raises candidates; the merge check defaults towarn, which blocks nothing. - Migration
0040createssource_writtenandsource_written_pending, empty. The first run of each Source after it is compare-only: every owned Attribute whose upstream value differs from the target's and was not written by that Source (an import, a hand edit, another Source) is aDV1Conflict under the defaultlocal_edits: review, and a snapshot deletion of a Record holding such values aDV2, so anauto_when_cleanSource that merged silently stops at an open Proposal until someone reviews it. Run each Source withdry_run: truefirst (local_edits,seeded). When the target's values are stale copies of upstream, setlocal_edits: source_winsfor that first run (the old behavior, each overwrite audited) and back toreviewafter; otherwise review in bulk withlorebase proposal conflicts PROPOSAL --status open --jsonandlorebase proposal resolve PROPOSAL --file resolutions.json(orPOST .../proposals/{p}/conflicts:resolve), then merge (operations). - Migration
0042addsproposals.kinds(null for existing Proposals until their next merge preview; meanwhile their Kinds are computed when steward roles, assignees or the steward approval rule are read). Withoutsettings.extractionand withoutstewardsin a schema nothing changes;scribe.extraction.mode: directis refused at startup (operations). - Enabling
store.resolve_misses.enablednow requiresstore.resolve_misses.caller_key(at least 32 characters, asecret://reference); the settings fail validation without it. - From 0.2.3: replace the images (or the packages) with 0.3.0, run the migrations to
0047(0041,0043,0045,0046and0047are additive), then restart Keeper and Scribe. Read the notes on0040and0042above before the first Source runs. - From 0.2.0, 0.2.1 or 0.2.2: as from 0.2.3; the migrations include
0037of 0.2.2. - From 0.1.0: follow the upgrade notes of 0.2.0 below, with the images of 0.3.0.