Repository navigation
Releases: Collegium-Siderum/bookrack
Release list
v0.13.0
Added
- An error the control plane could not classify is logged on the
daemon side. A handler failure matching no known error type, and
an ops error wrapping an untyped report, were answered as-32603
and recorded nowhere but in the client's reply. The daemon now emits
one warning for each —unclassified handler errorwith the method
name and the flattened chain,unclassified ops errorwith the chain
— so it reaches the rolling log,bookrack logs, thelogevent
channel and the diagnose bundle. The reply is unchanged, and a
failure the classifier judged a fault is not logged. - A failed queue job records why, in parts. The queue worker runs
the failure through the classifier the control plane uses for a write
RPC, and keeps the result on the row asproblem: summary, detail,
hint, andretryable, the same four parts the error envelope carries.
queue.list, thelast_finishedofqueue.tick, and therecent
rows of MCPsession.queue_statuscarry it;errorstays as it was.
The queue document moves to schema 7; a document from an earlier build
loads unchanged, and an earlier build refuses one this build has
written, naming both versions. On the command line,ingest --wait,
papers ingest --wait,intake ocr --waitandqueue waitname a
failed job by the recorded summary instead of the whole chain and
print each distinct hint under the summary line;queue list --state failedprints the hints under each cause header. Under--json, a
record whose failure was classified carriesproblem. - MCP
session.queue_statussays why a job failed, and can list one
state. Arecentrow carriesfinished_atonce the job has ended
anderrorwhen it failed, with a dependency source location
crate-relative as on the error envelope. The new optionalstate
argument lists only jobs in that state, under the same cap of ten,
newest enqueued first; the counts still cover the whole queue, so
{"state": "failed"}reaches a failure that has scrolled out of the
unfiltered window. A state no row can be in is refused as invalid
params, with the accepted names in the hint. bookrack queue waitre-attaches to queued work. It waits on
every job that is pending or running when the queue is read — jobs
submitted later are not waited on — and ends with the summary and
exit codeingest --waituses:5when any job failed or was
cancelled.--job <id>names jobs instead, by any part of their id,
and may repeat; a named job that has already finished is reported as
it ended, soqueue retryfollowed byqueue wait --jobreports a
failure even when the retried job finished first. A paused queue
with selected pending jobs, or a daemon without a queue worker, is
refused up front with exit2instead of waiting on work nothing
will run. Both are checked once, when the wait attaches;
docs/operating.mdlists what a wait does not see after that.- Terminal queue jobs can be retried. The new control-plane method
queue.retryflipsfailed,needs_ocr, orcancelledrows back
topendingin place — one row by its id or any part of it, or
every row in one state — clearing the previous run's error and timestamps, so the
worker re-runs them as fresh submissions. Nothing else in the queue
document moves, and a paused queue holds the rows untilresume.
bookrack queue retryis the verb: with no flag it retries every
failedrow,--state needs-ocror--state cancelledpicks another
state,--job <id>picks one row, and the human output lists the
ids it flipped. queue listcan show one state, capped, with failures grouped by
cause.--state <s>keeps the rows in that state and--limit N
shows at most N of them, oldest first; the per-state footer still
counts the whole queue, and a capped listing saysshowing N of M.
--state failedgroups its rows under one header per cause with the
group's size, so a long failed set reads as a handful of reasons
instead of a wall of rows. Under--jsonthejobsarray carries the
selected rows and the rest of the reply is the daemon's.distill buildshows which stage of which book is running. The
build ran each book's pipeline without a word until the book's stage
table printed. It now draws2/5 <slug> [3/7] <stage>on stderr,
naming a stage as it starts; with stderr not a terminal it appends
one such line per stage, and under--jsonor--quietit draws
nothing.distill lintanddistill verifyare unchanged.- A book's embedding pass reports how far it has got. While the
queue worker embeds a book,worker.progresscarriesstage: embed
withstage_progressset to the fraction of the book's chunks
written to the vector store — always for the first and the last
batch, throttled in between — soingestandintake ocrshow a
percentage during the longest stage instead of a bare spinner. Paper
jobs reportprocessingonly. - A write's reply names the library it acted on. Every control-plane
method that takes the write mutex — thevectors,stamps,
corpus,metadata,remove, anddryrunwrites on both sides,
andlibrary.fork/mount/unmount— answers withlibrary:
the registry name the call resolved to, which for an unnamed call is
the default pointer at that moment. The reply kept no trace of it, so
an operator had nothing but memory to tell which library a write had
landed on oncelibrary.set_defaulthad moved the pointer. A reply
that already carried fields keeps all of them;library.fork
reports the source library.papers metadatawrites end their
sentence withLibrary: <name>. ingest.cancelreturns the full id of the job it cancelled. The
method takes a prefix and answered{ ok: true }, so the caller
could not tell which job the prefix had matched; the reply now
carriesjob_id.
Fixed
-
Book-side reads on a library that has not ingested yet answer
instead of failing. The book catalog is created byinitor the
first ingest, so a root thatlibraries addregistered and nothing
wrote to has none; the book reads opened it read-only and surfaced
the SQLite open failure as an internal error. That took down
bookrack list,find,stats,categories,ocr-pending, the
per-bookshowfamily, the metadata listings and the audit trails,
and the matching control-plane methods and MCP tools, while
searchanddoctoron the same root already read it as empty.
Those reads now probe for the catalog first, the way the paper-side
reads have since 0.12: listings return an empty page, stats and
categories count zero, and a per-book read reports the intake as
unknown. Nothing is created on disk by looking. -
A registered library whose root carries no identity manifest is
refused, not served empty. With a library's root on a volume that
was not connected, the mount point was an empty directory that
passed the existence check, and the daemon came up serving it as an
empty library: searches answered nothing,statusshowed no counts,
and an ingest wrote a second catalog, corpus, and vector store into
the mount point, where the volume covered them the moment it came
back. A registry entry that caches auuidis now served only from
a root whosebookrack-library.tomlrecords that uuid. A root with
no manifest is refused at bring-up and bylibraries mount, with
the hint to reconnect the volume first or re-register the library
where it now is; a manifest recording another uuid is refused naming
both. Nothing is locked or written to a refused root. An entry with
no cached uuid is served as before, anddoctor'sregistryrow for
a cached uuid without a manifest now says to reconnect the volume
before re-registering, since re-registering against an empty mount
point would mint a new identity for a directory that is not the
library. -
An unknown library name is refused with the names that would
resolve as the hint.-32010 invalid librarycarried the
registry's names inside the message,no library named "x" (available: a, b), and nothing inerror.data. The message now
states only the failure, anderror.data.hintreadsUse one of: a, b.; against an empty registry it namesbookrack libraries register. The command line prints the hint under the summary line,
so the names are still shown. The MCP tools' refusal of an unknown
libraryargument carries the same parts. -
A dry run no longer announces a library change.
dryrun,
papers.dryrun, and thedry_runlegs ofremove,papers.remove,
corpus.rebuild,papers.corpus_rebuild,vectors.reembedand
papers.vectors_reembedended with alibrary.changedevent, so a
subscriber refreshed a library that had not changed. They still take
the write mutex for the length of the preview —daemon.statereads
writing,mcp.availabilitypauses and resumes, and a concurrent
write sees-32001 busy— but the event is published only by the
writes themselves. -
A session or data-root lock that the filesystem refuses is no
longer reported as held by another process.bookrack runand the
daemon's data-root lock read every failure of the non-blocking lock
call as a conflict: on a filesystem whoseflockfails for some other
reason (a network mount, some FUSE drivers) the error saidbookrack session already runningorbookrack data root already in use,
quoted whatever pid the lock file last recorded, and the callers that
branch on that verdict — the second-instance handoff, the daemon's
"serve the root unlocked" fallback,libraries remove --purge— took
the conflict branch. Only a lock held elsewhere is a conflict now; any
other failure surfaces as an I/O error naming the lock file and its
cause. The conflict me...
v0.12.0
Added
-
A daemon brought up on an unregistered root with an identity
manifest registers it and serves the registry.bookrack run --data-dir <root>(or the data-root variable, or the portable
layout) on a root the registry did not know served that root alone,
anonymously, however complete its manifest. Bring-up now records the
root under its manifest name, leaves thedefaultpointer as it
was, and comes up serving every registered library, as it would had
libraries addrun first. A root with no manifest, an unreadable
manifest, or a name another root already holds is still served
alone, with the reason in the daemon log.daemon.statusreports
the names a session registered asauto_registered, and
bookrack statusshows them on aregistered_at_startuprow; both
are empty on the next start, which finds the entry in place. -
-32018 STATE_UNUSABLE: a store this build cannot serve has its
own error code. A catalog or corpus written by a newer schema or
demanding a newer reader, an index without build stamps or with
stamps that disagree with this build, and a vector sidecar naming an
unknown ANN kind or a mismatched dimension all answered-32603,
which tells a client to report a bug, and the CLI exited 1 on them —
while every one of them has a next step that is the operator's to
take. Those nine conditions now answer-32018, bare or through any
pipeline wrapper, with the step inerror.data.hint: run a newer
build,bookrack corpus rebuild,bookrack vectors reset,
bookrack vectors rebuild, orbookrack stamps reconcile. The CLI
exits 2 on the code, as it does for caller input, through a variant
of its own that keeps the hint.docs/control-plane.mdadds the
code to the list, the write-class mapping, and the exit-code table. -
bookrack diagnosebundles now cover the paper stores, the
reference store, and the queue document. A bundle carried the book
pipeline's three stores and nothing else, so a fault in the paper
catalog, areference.dbfrom another schema, or a queue document
the daemon refuses to start on left no trace in it. The paper
catalog, corpus, and vector sidecar now land underpapers/in the
same shapes as their book-side sections;refs/summary.jsonrecords
each distilled book's build provenance — schema, parser, build time,
intake, entry and warning counts — and the entry and overlay totals,
without any bibliographic column;queue/queue.jsonis the daemon's
queue document re-serialised through the scrubber, so job paths are
redacted like every other path. A store that is missing or cannot be
opened records that inopen-error.json, as the book sections
already did; a queue document from a newer binary is reported as
refused, with both versions named, rather than as malformed. -
library.categories: the library-wide category distribution. A
new control-plane method and MCP tool return every category tag with
the number of books carrying it, most-used first, plus the books no
tag reaches and the registry total. It is the browse entry the
categoriesfilter oflibrary.find_bookslacked: that filter asked
the caller to name a tag without any way to see which tags the
library carries, and a library nobody has tagged answered every such
call with an empty page that read like "no such books". The
distribution shows the vocabulary before the filter is used, and an
untagged library reports itself as one — every book under
uncategorised. Reachable from the CLI asbookrack rpc call library.categories;find_books's tool description now points at
it.
Changed
-
An index-profile file from an earlier schema still loads. The
loader refused anyschema_versionother than its own, so the first
bump of that constant would have rejected every user-written profile
in both directions, with no migration to run. It now reads its own
schema and every earlier one, the way the library manifest does, and
refuses only a file from a later one, naming both versions. A bump
that changes the shape ships a reader for the previous schema in the
same change;docs/UPGRADE.mdcarries the row. -
bookrack --helpopens with the first run. The trailer named
run,rpc list, anddoctorbut neverbookrack init, the step
everything else assumes. It now says whatinitdoes and follows it
with the same three-command sequence the wizard's closing screen
prints, rendered from the same list, so the two surfaces cannot
drift apart in wording. -
doctorandlibrary.infostate what happened to a registry
default a path-class selection passed over. The data-root row
called the default "shadowed by" the flag or variable, which on a
daemon serving the whole registry is not so: an unnamed call still
reaches that default. Both surfaces now say which source selected
the root and where the default points; the row keeps its remedy. -
bookrack runs showreads itsprofiles:section from the run's
rollup instead of re-counting the audit tables. The rollup row a
run materialises at close (n_books,n_papers, the histograms)
never moved, but the profile buckets beside it were counted live on
everyshow. Both audit tables hold one projection row per item,
rewritten by whichever run judges the item last, so after a
glean --forceor a per-item re-audit the buckets of the earlier run
lost rows while itsn_papersdid not, and the two disagreed on the
same screen.compute_run_summarynow stores the buckets as a JSON
column onpipeline_run_summary, andruns showrenders that
snapshot. Rollups materialised before this build carry no snapshot;
for those the command still counts live and labels the section
profiles (counted now, not at run close)with a note that the
numbers need not add up.pipeline_run_idon an audit row is
documented as the pointer to the pass that judged the item last, not
as run membership. Catalog schema advances touser_version17. -
bookrack distill build --dry-runregisters its run as
distill_dryrun. A preview used to open apipeline_runsrow under
distill_build, sobookrack runs listshowed a rehearsal and a
build that wrotereference.dbas the same command, and a
--command distill_buildfilter counted both. The preview now
carries its own name, matchingdryrunandpapers_dryrun; it still
writes itsbook_distill_auditrows, andruns showstill renders
them. Existing rows keep the name they were written under. -
bookrack verifynow judges its report, and prints it for a
reader. Since 0.4.0 the command fetched the report over the control
plane, printed it as JSON, and exited 0 whatever it said — a catalog
whose schema failed verification and a healthy one returned the same
code, so the exit-code table gave scripts nothing to branch on, and
the renderer that used to draw the report for a person was never
called. The command now renders the report as text (the raw result
stays available under--json) and exits1when a finding says the
library is damaged: a store that does not verify or cannot be read
(catalog_schema_error,corpus_schema_error,intake_scan_error),
an unreadable vector sidecar (vectors_meta_error), one store missing
while the other is present, or an intake row whose file is gone
(missing_intake_files). An uninitialised root, a library that never
built its vector index, and any amount of churn exit0: not built
is not broken. The report is drawn before the exit code is set, so
the exit code adds no line of its own. Scripts that parsed the
default output as JSON pass--json. The exit-code table in
docs/control-plane.mdgains the source.
Fixed
-
The MCP
library.infocard describes the library it was asked
about. With several libraries served, a call carryinglibrary
took its counts from the selected library but its identity
(data_dir,library_name,resolution_source,
embed_model_configuredand the registry-identification fields)
from a snapshot of the library the daemon came up on, so one card
paired one library's counts with another's root. The MCP tool now
builds the card's identity from the selected library, as the
control-plane method already did; only the MCP listener address, a
property of the process, still comes from the snapshot. -
A running daemon stops returning the rows of a removed book.
The vector store handle the daemon keeps warm for search was opened
on a LanceDB connection that never checked for versions committed by
other connections, soremove,vectors rebuild, and every other
command that writes the store through its own handle left the search
path reading the table as it stood at startup. Status reads, which
open a fresh handle per call, reported the removal while search kept
serving the removed book's chunks with an empty breadcrumb, and only
an unmount and mount of the library brought the two into agreement.
The store now opens its connection with a zero read-consistency
interval, so every read resolves the latest on-disk version first. -
Paper-side reads on a library that has not gleaned yet answer
instead of failing. The paper catalog is created by the first
glean, so a library that has only ingested books has none; the paper
reads opened it read-only and surfaced the SQLite open failure as an
internal error. That took downbookrack listin its default scope,
bookrack stats, everypapersread, and the MCPlibrary.stats/
library.list_papersfamily. Those reads now probe for the catalog
first: listings return an empty page, stats count zero papers in
every status, and a per-paper read reports the intake as unknown.
Nothing is created on disk by looking. -
**
bookrack initreports a refusal as one line with a hint, a...
v0.11.0
Added
-
A running daemon mounts and unmounts libraries. The set a daemon
serves used to be decided once, at bring-up: a library registered
afterwards was invisible to it until a restart, and a root it held
could only be released by stopping it.bookrack libraries mount <name>opens a registered library and starts serving it;bookrack libraries unmount <name>stops serving one and gives its data root
back. Neither needs a restart.Mounting runs the same checks bring-up runs — a root another mounted
library already claims, an index profile that disagrees with the
served set on the reranker stage, an embed backend that cannot serve
— so a library that starts cleanly also mounts cleanly. It takes a
registry name, never a path: registering a root staysbookrack libraries add.Unmounting refuses three libraries, each because the daemon would
otherwise go on describing something it no longer serves: the
registry default, which is where every unqualified command goes; the
library the daemon came up under, which is whatbookrack status
reports it as; and a library with queue jobs still to run, which
would fail on their next pull. The root is released once any call
still using that library finishes, so it may come free a moment
after the command returns.This is what
bookrack libraries remove --purgeneeded: purging a
root a daemon serves no longer means stopping the daemon, only
unmounting that one library. -
A forked library is served without a restart.
bookrack libraries forkmounts the clone it just built, so the new library
answers on the same command. A clone the daemon cannot serve is
still built and registered — undoing it would delete data to report
a serving problem — and the command says so, naming the reason and
thelibraries mountthat finishes the job. The next steps it
prints are the operator's now; they used to be written to the
daemon's own terminal, where the person who typed the command never
saw them. -
bookrack libraries defaultreaches the running daemon. The
pointer has two homes — the registry file and the cache a daemon
seeds from it at start — and the command wrote only the file, so the
listing reported the new default while every unqualified command
kept reaching the old library. It now hands the change to the daemon
when one is listening; the daemon writes the same file and follows
the pointer immediately. With nothing listening it writes the
registry directly, as before. A library the daemon is not serving is
mounted rather than refused, so the pointer is no longer confined to
whatever happened to be mounted. -
Three verbs read across both pipelines.
bookrack list,
bookrack find, andbookrack searchread the book catalog and the
paper catalog together, so a page covers the library rather than one
pipeline.--scopenarrows any of them tobookorpaper, and
defaults to both; the per-pipeline namespaces stay and mean the same
thing.listandfindpage each side separately —--limitand
--offsetapply per side, and each side reports its own total.
findtakes the filters both catalogs share, plus the ones only one
of them has:--formaton the book side,--year/--venue/
--doion the paper side, each requiring a--scopethat names its
own side. A scope reaching the other side is refused before anything
is sent, naming the catalog that carries the column.searchranks
both corpora against one query and has no paging; a merged search is
not recorded in the retrieval sidecar, since its results span two
corpora that no single fingerprint describes.Under the rows is the library they came from.
--jsonis assembled
by the command rather than forwarded:itemscarries the rows of
every side that was read, each naming its own kind and the id that
addresses it. See Browsing and searching both
catalogs.bookrack papers findgains the three shared filters it was missing
(--contributor-role,--language,--status), so it and--scope paperare the same query. -
bookrack show <kind>:<id>reads one item, whichever pipeline it
belongs to. Books and papers are two catalogs numbering their
intakes independently, so101names one of each and a bare number
cannot address either without a namespace to fix it. An item id now
carries its own kind —book:12,paper:101,
reference:name_alpha/smith— and the top-levelshowreads
whichever the id names. It is the first read verb on the book side of
the external command line;bookrack papers showstays, and means
the same thing.A card names the library the record came from, so an item read on a
multi-library daemon is not left ambiguous. An id whose kind has no
read path yet says so and points at the surface that does carry it,
rather than at a command that would fail.The same
<kind>:<id>form is accepted everywhere abookrack paperscommand takes an intake id —paper:101as well as101—
and abook:12there names the other catalog and is refused while
arguments are read. It is also what listings print, so an id copied
out of one command can be pasted into the next. See Naming an
item. -
--librarynow travels with the call. The flag named a library
and then stopped at the CLI: every control-plane method that takes a
libraryparameter was reached without one, so the daemon resolved
the registry default and the flag survived only as an assertion that
the running daemon served the same library. It is now put on the
wire — underlibraryfor every routed method, and undernamefor
library.info, whose own parameter predates that spelling.A method that reports on the daemon rather than on a library —
status,doctor.gather,libraries list— cannot honour a
selection, and naming one on such a command is now refused with
the reason and what to run instead, rather than silently answering a
question about a different library. Methods that describe the
process (diagnose, the queue verbs,logs) are unaffected: a
selection there is meaningless, not wrong.A
librarythe caller wrote out by hand always wins, sobookrack rpc callstays an escape hatch. -
bookrack libraries listmarks which libraries the daemon
serves. The listing reads the registry file, so it shows every
registered library whether or not a daemon is running — and gave no
way to tell those two sets apart. Aservedcolumn now marks the
entries a running daemon holds; the sets differ whenever the daemon
came up on a root the registry does not name, because such a daemon
serves that root alone. With no daemon answering, the column is
absent rather than a column of noes: nobody was asked, which is a
different answer from nobody serving them. -
daemon.statusreports every library the daemon serves. A new
servedarray carries one row per mounted library —name,
data_dir,default,primary— so a client can see the whole
served set, which library an unnamed call resolves to, and which one
the daemon came up under. Those last two are separate bits: a daemon
started under a library that is not the registry's default carries
them on different rows. The existinglibraryanddata_dirfields
keep their meaning, the primary's identity. -
daemon.methodsreports how each method takes a library. Every
row gainsselection(routed/process/unrouted) and
library_key, sobookrack rpc list— and any other client — reads
which methods act on one library, and under which parameter, off the
daemon instead of keeping a list of its own. -
Every paper curation edit now leaves an audit row.
papers metadata set/clear/void/ack/approve/reject/
reopen/contributor-add/contributor-removeeach append one
metadata_auditrow to the paper catalog, carrying who made the
edit, the field, the value it replaced, and the reason when one was
given. Until now the paper side wrote the change and nothing else,
so a corrected title left no record that anyone had corrected it,
and the two questions the book side has always answered — who
edited this, and what did it say before — had no answer on a paper.The edit is attributed to the surface that made it, so an agent
editing over MCP and a curator editing at the CLI are told apart in
the trail rather than both recorded ashuman. Each call is also
logged inmcp_tool_callsunder its method name, the way paper
reads already were. -
The paper metadata audit is readable per field.
library.show_paper_metadata_reportrecomputes the plausibility
audit from a paper's cached extraction against its current effective
metadata and returns the whole judgement: per-field origin, grade,
flags and hint, the cross-field flags, and the CSL type the
required-field matrix was chosen by. The book side has had this read
since the audit existed; on the paper side the only way to see why a
record was gradedneeds_workwas to run a re-audit and watch the
two-scalar rollup move.Each response also carries the judgement stored on the paper's audit
row. The two disagreeing is the answer to a question nothing could
answer before — whether the paper has been edited since it was last
judged — andpapers metadata reauditis what closes the gap. -
The paper review queue is browsable.
library.list_paper_metadatalists papers with their confidence and
review status, andlibrary.list_paper_pending_reviewsis the preset
for the ones still needing a decision. The paper side had the four
review verbs but no way to ask which papers they applied to, so a
record entering the queue at ingest had no route out that...
v0.10.0
Added
-
retrieval:
showflags a recall set that is entirely weak.
bookrack retrieval show <call-id>now closes its hit table with one
advisory line when every recorded hit of the call sits at or above the
weak-match distance threshold, naming the threshold that made the
verdict so the line can be checked against the library's
configuration. The verdict is over the whole set — a single strong hit
vetoes it, and a call with no hits draws no line, because an absence is
not a weak result. This is the first consumer of
search.weak_threshold/BOOKRACK_SEARCH_WEAK_THRESHOLD: the knob
resolved through the full precedence chain but nothing read the
resolved value. The threshold comes from the libraryretrieval show
was pointed at, under the usual environment override. Nothing grades
an individual hit, and the search responses are unchanged. -
search: recall can exclude named books and papers.
library.search
acceptsexclude_book_intake_idsandexclude_paper_intake_idson
both the MCP and control-plane surfaces, dropping the named items'
passages server-side instead of leaving the caller to filter a result
set it has already paid for. The exclusion is a predicate on the ANN
query, so a caller that has read a dominant book can push it out of
the way and see what the rest of the library says. The two id name
spaces are unrelated, so each list applies only to its own side and
naming the side the call does not search is rejected rather than
ignored; an omitted or empty list filters nothing.kind="all"
carries both lists at once and applies each to its own store.
library.search_in_bookandlibrary.search_in_papertake no
exclusions — recall there is already confined to one item. The
recorder logs a non-empty exclusion set alongside the ANN knobs. -
reference: wildcard lookup can exclude books.
reference.lookup
acceptsexclude_books, a list of book slugs dropped from the
result whenbook="*"spans every registered reference book. The
exclusion holds across the whole resolution chain, including the
latin-key fallback retry and the redirect hop, so an excluded book
cannot re-enter as a redirect target. Passing the list with a
single-book scope is an error rather than a silent no-op. -
runtime: the daemon mounts every registered library. With the
primary root selected through the registry (--libraryor the
registrydefault), bring-up now warms a handle for each registry
entry: every library answers reads, queue jobs route to their
target library by name, and each served root's lock is held for the
daemon's lifetime (solibraries remove --purgeon any registered
root needsbookrack quitfirst). Two registry entries pointing at
the same root refuse startup naming both. A path-class root the
registry does not know keeps the previous single-library form.
Runtime mount/unmount and per-library status surfaces are later
milestones. -
cli:
bookrack status, a one-screen status card. One
no-argument command answers "is a daemon running, which library does
it serve, is it busy": with a live daemon it renders a single card
withdaemon.*/library.*/queue.*sections; with no daemon it
prints a short card and exits 0 ("not running" is an answer, not an
error); a held session lock whose control plane stays silent for 2s
exits 3 with the same stale-lock contract asbookrack run.--json
emits the card as one JSON object in every state;--quietanswers
through the exit code alone. The identity rows come from the daemon
over RPC, not from the lock file. Embedder and reranker health stay
withbookrack doctor; the card links to it instead of paying the
network round-trips. -
runtime: the
statusRPC reports the served library. The
response gainslibrary(registry name,nullfor a data root
selected directly by path) anddata_dir(the served root), so any
control-plane client can name what the daemon serves without reading
the session lock. Both are single-library snapshot fields; a
multi-library daemon will report them in plural form. -
config: registry writes are serialized by a sibling lock file.
Every registry write verb — set-default, upsert, repoint, remove —
now takes a short-held exclusive lock on<registry>.lockaround its
read-modify-write window before replacing the file atomically. The
daemon became a registry writer whenlibrary.forkand now
library.set_defaultpersist to it, so the single-writer assumption
behind bare atomic replacement no longer holds; the lock closes the
window where two writers' read-modify-write cycles would silently
swallow each other. Readers take no lock — the atomic rename still
hands them the old file or the new one. The lock file carries no
content and can be deleted at any time.
Changed
-
config:
mcp_addrandlog_directiveare retiredconfig.toml
keys. Neither ever resolved. Both are process-level knobs, read
before any data root is known — the log subscriber is installed and
the MCP listen address is chosen by the binary that then goes looking
for a library — so a value written into a library'sconfig.tomlwas
accepted, stored, and never consulted: the write succeeded, a daemon
restart changed nothing, and no surface said so. The two keys now join
embed_modelandindex_profileas retired: a file carrying either is
refused by name on every command that resolves a data root, and the
error gives the knob's real home (BOOKRACK_MCP_ADDRorbookrack run --mcp-addr;BOOKRACK_LOGandBOOKRACK_LOG_CONSOLE) alongside the
libraries config <name> --unset <key>that deletes the line. Both
environment variables are unchanged.libraries config <name>also
annotates a retired line when it prints a file verbatim — the one read
surface such a line used to pass through unremarked. Upgrading a
library that still carries either line requires the unset; see
docs/UPGRADE.md. -
cli: a caller without a terminal is asked to confirm, not
refused.vectors drop,vectors reset, their twopapers
peers,index-profile apply, and thelibraries adduuid-clash
resolution refused outright when stdin was not a TTY. That predicate
answers "is a human at a terminal", not "can an answer arrive", and
the two differ often enough to matter: this repository's own
end-to-end tests drive the strongest gate in the product — retyping
a library name beforelibraries remove --purge— over a pipe, and
ssh host …without-t,docker execwithout-t, and Git Bash
on Windows all put a human behind one. Refusing them pushed
operators to--yes, which skips the retype entirely. All three now
prompt like every other destructive verb, under the answer window
BOOKRACK_CONFIRM_TIMEOUT_SECSbounds (see Fixed). A caller that
supplies no answer still fails with the same
directed message and the same exit2; what changed is that a
caller that does supply one is now heard.DestructiveGateis gone:
its stdin axis moved torender::confirm'sanswer_window, and
keeping it would have been a second copy of one rule. -
extract: a fullwidth References heading terminates the paper
metadata-scan window.extract_paper_metadata_textdocumented its
References-heading match as running against the fullwidth-folded
page text, but the fold was applied only after the window was cut,
so a heading printed in fullwidth forms (U+FF01..U+FF5E) never
closed the window and bibliography text leaked into the DOI / venue
scan. Each page is now folded before the heading match, making the
documented order the real one. -
glean, catalog: three paper-audit grade columns no longer stuck
atmissing. The audit's per-field reports and the
node_paper_auditprojection disagreed on field-key names:
GRADE_COLUMNSlooked uparxivandcontainerwhere the graders
recordarxiv_idandcontainer_title, sograde_arxivand
grade_containerwrotemissingregardless of the paper; and the
abstract grader read the nonexistent effective keyabstractwhere
the effective view names itabstract_text, sograde_abstract
always gradedmissingand theabstract_too_shortsignal could
never fire. The keys are now unified on the effective-attrs names
end to end (graders, roll-up candidates, CSL requirement matrix,
projection). Audit rows written before the fix carry the stuck
missinggrades until their papers are re-audited
(papers.metadata.reaudit). -
glean: a thesis with its institution can audit clean. The CSL
requirement matrix markspublisher(the institution, in CSL
terms) as required for a thesis, but no grader produced that field,
so the roll-up saw it as missing on every thesis and floored the
verdict toneeds_workwith low confidence regardless of the
actual metadata. A presence-only publisher grader now feeds the
roll-up; the grade travels in the report JSON and has no dedicated
grade_*column onnode_paper_audit. Thesis rows audited before
the fix stayneeds_workuntil re-audited
(papers.metadata.reaudit). -
distill: the
splitscoverage counter is actually written. The
Coveragedoc promised its counters were written by the matching
stages, but neither splitter stage touchedsplits, so every
book_distill_auditrow storedsplits = 0regardless of the run.
Both splitters now record their output count; audit rows written
before the fix keep the zero until their book is re-distilled. -
glean:
venue_aliasesis actually consulted. The paper-audit
data set documentedvenue_aliasesas a map of informal-to-canonical
venue spellings, but no grader read it:grade_containerchecked
the whitelist against the raw value only, so an aliased spelling of
a whitelisted venue was still fla...
v0.9.0
Added
-
config, cli: the library manifest carries the index-profile
reference; the registry entry becomes a regenerable cache of it.
bookrack-library.tomlgains an optionalindex_profilekey, and
declaring a profile — throughindex-profile applyorlibraries config <name> index_profile=<p>— writes it there in one atomic
write, then refreshes the registry cache and sweeps a superseded
config.tomldeclaration. The reference now travels with the data:
copy a root to another machine,libraries registerit, and its
profile comes along;libraries scanrefills the cache for every root
it finds. Resolution reads manifest, thenconfig.toml, then the
registry entry, so references declared the old way keep working
untouched — seedocs/UPGRADE.mdfor the one-command
migration.format_versionstays at 1: the key is optional and
additive, and unknown keys were always tolerated. -
session, runtime: data roots are guarded by an exclusive
.bookrack.lock. The daemon takes the new root lock during
bring-up — after resolving its configuration, before opening anything
under the root — and holds it until shutdown, so a second daemon
pointed at the same data root fails to start instead of racing the
first one through SQLite, LanceDB, and the queue snapshot. This
closes the gap that two differentBOOKRACK_RUNTIME_DIRvalues left
open: the session lock only ever bound one daemon per runtime
directory, never one writer per data root. Offline read-only commands
(distill,runs,retrieval,doctordiagnostics) do not take the
lock and keep working alongside a serving daemon; a read-only data
root, which cannot host a lock file at all, is served unlocked with a
warning. -
runtime:
workingdaemon state, and the lifecycle state is now
derived instead of assigned.daemon.state/status.stategain a
fifth value,working: the queue worker is executing a job, without
the write-mutex and MCP-pause semantics thatwritingimplies.
Internally the single state flag is folded from independent activity
sources (RPC write session, executing queue jobs, degraded
conditions, shutdown) by precedencestopping > writing > working > degraded > idle, so overlapping activities can no longer clobber
each other's transitions — an RPC write finishing while a queue job
still runs falls back toworking, notidle. -
mcp, runtime:
library.show_toc/library.show_paper_tocare
paginated. Both TOC reads accept optionaloffset/limit
parameters (the limit defaults to and is clamped by the 2000-entry
cap) and returntotaland anext_offsetcursor alongside the
existingtruncatedflag. A TOC larger than one page is now
reachable page by page — previously everything past the cap was
silently unreachable. Requests without the new parameters return the
same entries as before. -
mcp, runtime: TOC reads gain
titles_onlyandmax_depth
projections. Withtitles_only, each TOC entry shrinks to
node_id/title/depth— enough to scan a book's structure
and address a follow-up span read at a fraction of the response
size.max_depthkeeps only the shallow levels (the book root is
depth 0, somax_depth: 1is the root plus its top-level
divisions). Both compose with pagination, andtotalcounts the
filtered set. -
mcp, runtime:
library.show_book/library.show_papercarry
toc_stats. The detail responses gain an aggregate of the
ingested TOC —entry_count(the book root included) and
max_depth— ornullwhen the item has no corpus nodes, so a
caller can pick ashow_tocpagination or projection strategy
before pulling any entries. Theshow_metadata_auditresponse
embeds the same detail shape and carries the field too. -
mcp, runtime: TOC reads gain a
title_substringfilter.
library.show_toc/library.show_paper_toccan keep only entries
whose title contains a substring (ASCII case is ignored;LIKE
metacharacters match literally), so locating one chapter by name is
a single filtered call instead of a full TOC pull. Composes with
pagination and the projections;totalcounts the filtered set. -
config, cli:
config.tomlgainsindex_profileand a[search]
table. A library's per-root config can now name the index profile it
runs under and pin the two retrieval knobs (search.top_k,
search.weak_threshold) that were previously env-only.bookrack libraries configaccepts the three new keys — dotted keys edit the
nested[search]table, numeric values are written as TOML numbers,
and anindex_profilereference is refused at write time when it does
not resolve or fails static validation. The resolution chains are
pinned:BOOKRACK_EMBED_MODEL> explicitembed_model> the
profile's model > default, andBOOKRACK_SEARCH_*>[search]>
default. -
index-profile, runtime:
index-profile currentandindex-profile diff.current [--library <name>]reports, offline and read-only,
the profile a library effectively runs under — the reference's origin
(manifest,config_toml, orregistry), any lower-priority source
still naming a different profile (drift), the resolved
embed/ann/reranker combination, the effective embed model, and a
field-by-field comparison against the built index stamps.diff <a> <b>compares two profiles field by field. Both take--json. -
index-profile, runtime, cli:
index-profile applyreconciles a
library with a profile. The high-level path for switching embedding
models or ANN parameters: statically validate the target profile,
refuse whenBOOKRACK_EMBED_MODELor an explicitembed_modelfield
would mask it, compare the recorded stamps and the persisted ANN
configuration per pipeline (books and papers derive independently;
an unused pipeline is skipped), and derive the action list — a
destructivereset(followed by therebuildthat realizes the
declared index) when the embed model or dimension changed, a
reembedfor a chunking/normalization version bump, arebuildor
dropthat realizes the declared ANN state, astamps reconcile
for missing stamp records. Applying to an already-consistent library
still writes the profile reference.
--dry-runprints the plan offline; executing routes every action
through the daemon queue in order, one result line per step. A plan
containing a reset demands the library name retyped (or--yes), and
the recommended rehearsal islibraries fork→ apply on the copy →
verify → apply in place. The target declaration is written to the
library manifest after confirmation and before the first action, so an
interrupted run reports "declaration ahead of stamps" incurrentand
a re-run re-derives only the remaining actions. -
config, runtime, cli:
bookrack doctor --install-rerankerinstalls
the pinned reranker artifacts. The reranker backend's two native
artifacts — thellama-serverbinary (llama.cpp releaseb9934,
unpacked whole so the executable keeps its dynamic libraries beside
it) and theQwen3-Reranker-0.6BGGUF model — are now pinned by
SHA-256 in the config crate (LLAMA_SERVER_VERSION.mddocuments the
pair) and installed into per-user managed directories by the new
doctor verb, which downloads whichever is missing and is a no-op
when both are in place. Two new escape hatches,
BOOKRACK_LLAMA_SERVER_BINandBOOKRACK_RERANKER_MODEL, name the
artifacts directly and are authoritative when set. -
rerank, runtime, config: the reranker backend — a supervised
llama-server subprocess with an operator-URL escape hatch. A new
rerankcrate wraps llama-server's/v1/rerankendpoint behind the
same typed error contract as the embed client. When a library's
effective profile enables a reranker stage, the daemon now brings up
a verified backend before serving: by default it spawns the pinned
llama-serveron a loopback port, holds startup until/health
reports the model loaded, restarts it with backoff if it crashes,
stops it TERM-then-KILL at shutdown, and kills a recorded orphan
left by a killed daemon. Settingreranker.urlinconfig.toml(or
BOOKRACK_RERANKER_URL) switches to an operator-run server, probed
once at startup instead of spawned. The[reranker]table also
carriesctxandthreadsfor the supervised server, all three
keys writable throughbookrack libraries config.bookrack doctor
grows a reranker section (binary, model, backend status) that
appears only when the effective profile enables the stage. -
index-profile, ops, runtime: the reranker stage is live —
qwen3-4b-qualitybecomes a runnable profile. A profile whose
[reranker]section names a cross-encoder no longer refuses
startup: the daemon brings up the verified backend, and every search
op then recalls the profile'stop_k_incandidates, scores them
through the reranker, and returns at mosttop_k_outpassages
ordered by relevance, each citation carrying arerank_score
alongside its ANN distance (the field is absent for reranker-less
libraries, so their wire shape is unchanged). The unified
book+paper search reranks the merged candidate window, so the stage
scores one profile-sized set regardless of how candidates split
across stores. A stage failure fails the search — the profile
promises an atomic retrieval combination, and under the supervised
backend an outage is a self-healing restart window — rather than
silently returning the unreranked order. The built-in
qwen3-4b-qualityprofile is finalized to the implemented backend
(llama-server, candidate window 50 -> 10), the validator now
rejects any other reranker backend, andindex-profile current/applydrop their not-implemented notes;applyhints at
the daemon restart that...
v0.8.0
Added
-
index-profile, runtime, cli:
bookrack index-profilenames and
statically checks a retrieval configuration. A newindex-profile
crate couples the three retrieval knobs — the embedding model, the ANN
index shape, and the reranker stage — into one named, validatable atom,
with two built-in presets compiled into the binary
(qwen3-0.6b-default, a product-quantized IVF index with no reranker;
qwen3-4b-quality, an HNSW index with a cross-encoder reranker stage).
Three read-only verbs resolve locally with no daemon:index-profile listmerges the built-ins with any user profiles under the per-user
index-profiles/directory (marking each[builtin]/[user]and
noting shadows),index-profile show <name>prints a profile's source
and its validation result, andindex-profile validate <name>runs the
static checks and exits non-zero on any error. Validation enforces the
product-quantization constraints (num_sub_vectorsrequired, dividing
the dimension, withdim / num_sub_vectors <= 8), warns on the
HNSW recall regression, checks the cross-encoder reranker contract, and
consults an offline model registry that--allow-unknown-model
bypasses. A user profile shadows a built-in of the same name. -
runtime:
bookrack doctorchecks registry–manifest consistency and
index-profile coherence. Two new row groups. The first walks every
registry entry and compares it against its on-disk identity manifest,
warning when the cached uuid is stale, when the data root is missing (a
move or an unmounted volume), or when the manifest is unreadable — a
name alias and a kind override stay legal and are not flagged. The
second, for every entry that records anindex_profile, resolves and
statically validates the profile and compares its embed model and
dimension against the library's built index stamps, warning when a
referenced profile is undefined, invalid, or disagrees with the built
index (which would make the daemon refuse to start). Both run on the
daemon-side and daemon-free doctor paths alike; the coherence check
opens the corpus read-only and skips silently when it cannot. -
config, cli: offline
libraries configreads and edits a library's
config.toml.libraries config <name>resolves the library's data
root from the registry with no daemon and prints itsconfig.toml
(the parsed knobs under--json, the raw file otherwise);
libraries config <name> KEY=VALUE ...and--unset KEYedit the
file in place, preserving hand-written comments and layout. Accepted
keys are the fourRootConfigknobs —ollama_url,embed_model,
mcp_addr,log_directive— and an unknown key or a malformed
ollama_url/mcp_addrvalue is refused (exit 2) before anything is
written. A missingconfig.tomlis created on first write. Editing
embed_modelwarns that re-ingestion is required; a set value already
shadowed by an environment variable, and the need to restart a running
daemon, are noted.configexposesset_root_config_values,
read_root_config_text,ROOT_CONFIG_KEYS, and
root_config_env_override. -
config, ops, runtime: a path-resolved root is identified against the
registry. When the data root is won by a path — the--data-dir
flag,BOOKRACK_DATA_DIR, or the portable layout —confignow
matches it back against the registry, by the root's manifest uuid
first and then by path, and claims the registered library name instead
of leaving the root anonymous.bookrack inforeports how the name was
identified (manifest uuidorpath), andbookrack doctorappends
the same fact to itsdata rootrow; a registry selection reports
nothing extra, since the source already conveys it. TheLibraryInfo
DTO gains an optionallibrary_identificationfield, andconfig
exposes theLibraryIdentificationenum and afind_library_by_uuid
helper. The precedence itself is unchanged; a path-class daemon that
previously reported the placeholder namedefaultnow reports the
registered name. -
config, cli, runtime: a shadowed registry default is now visible.
When a path-class source wins the data root — the--data-dirflag,
BOOKRACK_DATA_DIR, or the portable exe-neighbour layout — a registry
defaultthe operator set still applies on disk but has no effect on
which root is served.confignow computes that eclipse once and
surfaces it three ways:bookrack infoadds ashadowed_defaultline
(registry default '<name>' is shadowed by <source>),bookrack doctorturns thedata rootrow from OK to WARN with the same fact
and the remedy (unset the variable or pass--library <name>), and
libraries default <name>warns on stderr whenBOOKRACK_DATA_DIRis
set to another root and will shadow the default just written. The
precedence itself is unchanged; only the diagnostics are new. The
LibraryInfoDTO gains an optionalshadowed_defaultfield. -
config, cli: offline
libraries add,register, andremove, plus
scan --register. Three write verbs bring an existing data root into
the registry with no daemon.libraries add <name> <path>registers
under an explicit name;libraries register <path>derives the name
from the root's manifest (or its directory), taking--name <alias>
when the derived name is taken. Both write a fresh identity manifest to
a root that has none — previewed and confirmed first, unless--yes—
and degrade to a uuid-less entry rather than failing on a read-only
volume. A registration whose identity uuid already belongs to another
entry stops for a decision: interactively, move the existing entry to
the new path; non-interactively, exit with the exact move/--new-uuid
commands.libraries remove <name>forgets an entry and keeps the
data;--purgeadditionally deletes the data root behind a detect gate
and a typed confirmation.libraries scan --registerregisters every
confirmed root it finds — turningscan --volumes --registerinto a
one-command registry rebuild after a reinstall — and skips probable
roots with a note to add them by hand. -
config, cli: offline
libraries detectandlibraries scan. Two
read-only, daemon-free subcommands answer "is this path a bookrack
data root?".libraries detect <path>returns a three-way verdict —
confirmed (a readable identity manifest), probable (no manifest
but thecatalog.db+corpus.dbpair is present), or not a
library — reporting an unreadable manifest (foreign magic or a future
schema version) as its own distinct outcome; it exits 0 for
confirmed/probable, 1 otherwise, and 2 for a missing or non-directory
path.libraries scan <parent>walks a parent directory's immediate
children (or, with--volumes, mounted volumes two levels deep),
lists the data roots found, prunes at each hit, and always reports how
many entries it skipped. Both honour--json. -
config, runtime: data roots carry a self-describing library
manifest. Every data root now holds abookrack-library.toml
naming the library and its stable UUIDv7 identity (format,
format_version,uuid,name,kind,description,
created_at), so a library describes itself and the registry becomes
a regenerable cache over the manifests rather than the sole record of
identity.bookrack initwrites one for the new root and caches the
uuid into the registry entry;libraries forkwrites the fork its own
manifest with a freshly generated uuid. A root without a manifest
stays valid, and a v1 reader tolerates unknown keys within v1.x. -
config, cli, runtime: the library registry carries per-library
metadata andlibraries defaultpersists. Registry entries may
now be metadata-bearing tables ([libraries.<name>]withdata_dir,
kind,description,index_profile,created_at,uuid)
alongside the legacy bare-path form, which stays permanently
readable and is treated as{ data_dir = "<path>", kind = "prod" }.
Any write rewrites the file into the table form atomically
(same-directory temp file plus rename) and prints
info: registry upgraded to entry-table formaton stderr the first
time it upgrades a legacy file.bookrack libraries default <name>
now resolves locally and writes the on-diskdefaultpointer
directly, so it needs no running daemon and the choice survives a
restart; naming a library the registry does not define exits 2. The
library.set_defaultRPC remains but is documented as affecting only
the running daemon session.libraries listgains akindcolumn
(andkindfield under--json). -
corpus, catalog, ops, cli: every single-store search invocation is
recorded against the corpus state that served it. A new
Corpus::compose_corpus_fingerprinthashes the five corpus stamps
(embed model, vector dimension, chunk version, normalize version,
and the ANN kind fromvectors_meta.json,brute-forcewhen no
meta file exists) into a 16-hex prefix. M[15] adds the sidecar pair:
retrieval_calls, one row perlibrary.search/
library.search_in_book/library.search_paper/
library.search_in_papercall carrying that fingerprint, the
requested depth, and the query, andretrieval_call_hits, one row
per returned passage keyed(call_id, ord)with its
norm_chunk_sha256and distance. The recorder writes the log row
and the sidecar in one transaction
(Catalog::record_tool_call_with_retrieval) and stamps the
fingerprint intomcp_tool_calls.extras; the unified two-store
search records no sidecar because one row cannot carry two
fingerprints.Catalog::prune_retrieval_calls_with_stale_fingerprint
/prune_retrieval_calls_older_thantrim the tables (no CLI entry
yet);bookrack retrieval list [--last N] [--corpus-fingerprint HEX]
andbookrack retrieval show <call-id>r...
v0.7.0
Added
-
Reference-book pipeline (preview). A new
bookrack-distill
stage runner andbookrack-refsread store land the third
bookrack item kind alongside books and papers: aReference
variant oncore::ItemKind, areference.dbper library with the
baseline schema (reference_books,reference_entries,
reference_entry_overlays, areference_entries_resolvedview
that merges payloads throughjson_patch, and an FTS5 trigram
sidecar), the staged transformSource → Pages → Blocks → Raws → Splits → Draftsbuilt from segment / splitter / extractor /
finalize stages, and the three controlled vocabularies (property,
quality-flag, stage catalogs) with a cross-catalog validator. The
surface lands as threebookrack distill {build,verify,list}CLI
subcommands and two MCP toolsreference.lookupand
reference.overlay_set.This is a preview. The feature is not finished:
- OCR Markdown is the only supported input. The distill
pipeline reads the polyocr Markdown product of one reference
book; born-digital sources are not yet wired in. - Operators must author a
book.tomlper book. The pipeline
is config-driven: anchor rules, stage chain, property writes,
and lookup indexes are all declared in a per-book TOML file.
There is no auto-detection. The expected schema is documented
by the controlled vocabularies shipped under
crates/distill/data/; mis-shaped configs surface as
ParseError::CatalogViolationat load time. - No queue integration. A
Referencejob submitted to the
queue worker today panics the spawn-blocking task with
unreachable!; the CLI runs distill locally in-process
instead.
Treat the surface as unstable until a follow-up release lifts the
preview marker. - OCR Markdown is the only supported input. The distill
-
CLI shared render layer behind three new top-level flags:
--jsonswitches every reading subcommand from the default
human renderer (tables, key-value cards, one-line summaries) to
the daemon's structured payload;--quietsuppresses progress
spinners and confirmation chrome for non-interactive use; and
--no-colordisables ANSI styling even when stdout is a tty. The
flags resolve once into a process-wideRenderCtxso individual
subcommands stay free of mode plumbing. Reading subcommands that
previously dumped JSON only now branch on the context; writing
subcommands respect--quieton their progress and confirmation
surfaces. -
The session lock records the data-root path the daemon is
serving as a newlibrary_root=<abs path>line. The line is
append-only (Phase 1's "unknown keys ignored" rule still
applies), so an older client reads the file unchanged. The CLI
warns on stderr when the shell's intended library selection
(--data-dir/--library/BOOKRACK_DATA_DIR) disagrees with
the running daemon's recorded root, so an operator does not
silently send writes against the wrong library. -
CLI subcommand help is filled out across the surface: every
bookrack <subcommand>carries a one-line summary and a long
help body, and the global flags (--data-dir,--library,
--audit-profile) move under a shared Common Options group
so they no longer scroll past the subcommand's own arguments.
Argument-level help is filled in oningest,remove,
vectors, and thepapersaction arms. -
intake.ocrcontrol-plane method and the matchingbookrack intake ocr <ocr_md> --from-pdf <pdf>top-level CLI subcommand. The handler
enqueues a single OCR-intake job onto the persistent ingest queue;
the worker dispatches it as a book ingest whose source is the OCR
markdown product paired with the scan PDF anchor, by way of a new
LibraryHandle::ingest_ocrrunner. -
bookrack queuetop-level CLI subcommand covering the existing
control-plane methods that previously had no one-shot entry point:
queue list(→queue.list),queue pause/queue resume/
queue clear(→ the matchingqueue.*RPC), andqueue cancel <job-id-prefix>(→ingest.cancel). The sharedQueueAction
grammar gainsListandCancelvariants. -
bookrack libraries info [--name <NAME>]andbookrack libraries default <NAME>round out thelibrariessubcommand:infoprints
the per-library status card the daemon serves vialibrary.info,
defaultmoves the registry's default-library pointer through
library.set_default. Both RPCs were already on the control plane;
this lifts them out of thebookrack exec library.*catch-all into
first-class top-level commands. -
bookrack logsis now a top-level subcommand for the daemon's
log stream.--tail Nsnapshots the most recent N events via
thelogs.tailRPC and exits;--follow(the default with no
other flag) subscribes to the broadcast and streams new events
as they arrive; both together emit the snapshot first and then
switch to the live stream.--level TRACE|DEBUG|INFO|WARN|ERROR
(case-insensitive) drops every event below the given severity.
Human mode renders each event asHH:MM:SS LEVEL target | message;
the top-level--jsonflag emits newline-delimitedLogEvent
records sojq -cpipelines stay clean. The existing
bookrack exec logs follow|tail [<n>]keeps working as an
unsurfaced compatibility alias.
Removed
bookrack replstandalone control-socket REPL client and the
in-process reedline REPL hosted bybookrack run(including its
--legacy-repltransition flag). Operators reach the daemon
through one-shot subcommands,bookrack exec <method> '<json>'
for ad-hoc control-plane RPCs, the desktop tray, or MCP. The
cmd::repl_clientmodule, thetests/repl_e2e.rsintegration
test, thereedlineandshlexworkspace dependencies, and the
ReplCli/ReplCommandwrapper types from the former
bookrack-repl-grammarare deleted; the surviving crate is
renamed tobookrack-cli-grammarand keeps its leaf
clap::Subcommandtypes (IngestArgs,WriteMetadataAction,
WriteVectorsAction,CorpusAction,StampsAction,
QueueAction,RemoveArgs,DryrunArgs,IntakeAction,
Papers*) for the top-level CLI to consume.
Changed (breaking)
corpus.rebuild,vectors.reembed,remove, and their
papers.*peers now requireplan_idon the execute leg. The
transitional unpinned fallback that ran the legacy path with a
deprecation warning is removed; a missingplan_idreturns
-32602 INVALID_PARAMSpointing at the dry-run leg. All
in-tree clients (one-shot CLI subcommands, the tray) already
drive the two-step pinned protocol.
Changed
- Every predictable CLI failure now flows through a typed
bookrack_cli::error::BookrackCliErrorenum and the same
color-eyrereporter, ending the old split between
eprintln!+std::process::exit(2)for daemon-not-running, an
ad-hoc lock-conflict path inbookrack run, anddoctor's
hand-rolledstd::process::exit(1). Variants
(DaemonNotRunning,DaemonUnreachable,StaleSessionLock,
SessionLockUnreadable,DoctorUnhealthy) each carry their
own exit code and one-line message; unexpected errors still
fall through to the fullcolor-eyrecause chain so internal
bugs stay debuggable.helpers::connect_or_exitis renamed to
helpers::connectand returnsResult<Arc<ControlClient>>;
every call site uses?for propagation. The
oneshot_subcommands_consistent_no_daemonintegration test
continues to expect exit 2 and abookrack daemon not running
hint on stderr. - Destructive-action confirmation runs through a single
bookrack_cli::render::confirm::confirm_destructivehelper with
two strengths:Softaccepts a case-insensitiveyes/y
(used bycorpus rebuild,remove,vectors reembed, the
papers.*peers, andlibraries fork);Hard { token }
requires the operator to retype the literal sentinel (used by
vectors resetfor theRESETtoken). Prompts move from stdout
to stderr so a piped stdout stays clean, andlibraries fork
now printsForked library to '<name>' at <path> (<mode>).on
success instead of dumping the daemon's JSON response. The
legacycrate::util::confirm/confirm_token/
confirm_vectors_resetare removed. bookrack queue list,bookrack libraries list,bookrack libraries info,bookrack papers list,bookrack papers find, and
bookrack papers showrender a human table or key-value card by
default instead of the daemon's raw JSON. The top-level--json
flag restores the legacy pretty-printed payload for scripts.
queue listshows an 8-character UUID prefix in theidcolumn;
queue list --longprints the full UUIDv7 for copy-paste.
papers list/papers findcollapse to columnsid,title,
author,year,containerwith…truncation, and append a
(showing N of TOTAL; ...)hint when the result set is truncated.bookrack ingest <path>,bookrack papers ingest <path>, and
bookrack intake ocr <ocr_md> --from-pdf <pdf>wait for the
enqueued job(s) to reach a terminal state by default and print a
one-line human summary on stdout (Ingested <basename> as <id8> in 12.4s (done),OCR-ingested <basename> ...) instead of
exiting at queue-ack with a JSON job-id list. The historical
immediate-return behaviour is available under--no-wait, and the
top-level--jsonflag switches the wait-summary to a
JobOutcomeReportpayload for scripts. The CLI subscribes to the
event stream before submitting the RPC so a fast worker cannot
race past the wait loop's firstrecv.- The persistent queue document schema bumps to
v4:QueueJobgrows
anintake_ocrsidecar that carries thefrom_pdf,expected_pages,
andallow_partialfields the OCR ingest path needs. The field is
optional and defaults to absent, so av3document loads unchanged
and reads as a plain book ingest. The upgrade is one-way — ...
v0.6.1
Fixed
bookrack-runtimeraises its crate-levelrecursion_limitto
256so the trait solver can proveSendon the
methods!-generated dispatch future undertokio::spawn. The
macro collapses 30+ control-plane handlers into one async match;
combined with the layeredportable_atomicshim pulled in
transitively throughlanceon targets without native 128-bit
atomics (Linux x86_64 withoutcmpxchg16b), the default limit of
128 was just past the depth required to walk every handler's
future and every wrapper type. Targets with native 128-bit atomics
(macOS aarch64, macOS x86_64 with the feature) stayed under the
default limit, which is why local builds passed while CI's Linux
job failed withE0275. Zero runtime cost; the knob only widens
the compile-time search budget.
v0.5.0
Added
-
Two-store library: a
paperscluster (papers_catalog.db,
papers_corpus.db,lancedb_papers,papers_dir/) sits beside the
existing book cluster under the same data root, with parallel
intake / corpus / vector / metadata schemas. Theconfigcrate
exposes paper-side getters; the in-processLibraryRegistryowns
both stores and routes work by item kind, so books and papers share
one scheduler, one MCP surface, and one search call. -
bookrack-glean: paper-side pipeline crate covering
REGISTER → IDENTIFY → EXTRACT → STRUCTURE → CHUNK → EMBED. Driven
through the control plane byglean.submit, which the queue worker
consumes as paper-kinded jobs. REGISTER archives the source PDF
bytes next to the envelope and stampsintake.source_pdf_path;
IDENTIFY runs an offline pass that lifts DOI, arXiv id, venue, and
abstract from the cached extraction; EXTRACT reuses the shared PDF
adapter; STRUCTURE runs the paper-only coloring + Section tree
pass; CHUNK and EMBED reuse the book-side primitives against the
papers vector store. -
bookrack paperssubcommand tree (one-shot CLI and REPL):papers ingest [--recursive]walks.pdfdirectories and forwards each
path toglean.submit;papers sourcestreams the archived source
PDF back frompapers_dir/;papers removecascades a paper out
of the papers catalog, corpus, vector store, andpapers_dir/;
thepapers <read>commands mirror the book-side read family
against the paper backend. -
MCP surface for papers:
library.searchaccepts akindswitch
(book/paper/ unset for both);library.read_context/
library.read_spanand the other library read tools accept a
kindfield on their arguments; thepapers.*read family,
papers.fetch_source, andglean.submitexpose papers to agent
clients. -
CSL-JSON item model with two-way conversion in
bookrack-catalog.
BiblioandContributorgain paper-shaped fields (DOI,
container-title, volume / issue / pages, publication date parts,
…) and round-trip through the M8 paper columns without lossy
detours. -
KindedNodeId: every cross-store addressing surface
(read_context,read_span, search citations, queue jobs) now
carries the item kind alongside the intake id, so a hit, a context
read, or a queued job is unambiguous between the two stores.
library.searchcitations report the store they came from. -
Extraction envelopes are written with an explicit kind prefix in
their filename;bookrack-extracttolerates the un-prefixed legacy
layout on read.bookrack doctor --rename-envelopes [--dry-run]is
the one-shot migrator that rewrites existing envelopes into the new
naming. -
glean_paperSTRUCTURE now assembles a Section tree from the
heading-colored block stream: aBlockKind::Heading{1}opens a
Section organizer under the Work root, aHeading{2}opens a
Subsection (auto-opening a Section first when none is outstanding),
depth-3+ Heading blocks stay as Heading leaves under the deepest
open organizer, Body blocks attach as Paragraph leaves, and
BlockKind::Captionblocks land asFigureCaptionstructural
leaves. The abstract leaf is bit-for-bit unchanged (Tier 1 vector
anchor): same NodeId allocation order, sameintake:{id}:abstract
stable anchor, same text / norm hashes, and the page bounds stay
NULL. Body-leaf stable anchors continue to count Body blocks only,
so a re-glean of a Phase-1 envelope still produces the same body
hashes. When the heading pass identifies no candidates the tree
falls back to the flat Phase-1 shape. The STRUCTURE audit row's
metric_summaryJSON gainssections,subsections, and
headingscounters alongside the existingbody_leaves. -
bookrack_extract::pdf_paper::extract_paper_structured: a paper-only
coloring pass that promotes [BlockKind::Body] blocks to
BlockKind::Heading/BlockKind::Captionusing the PDF outline that
the adapter already attached to the extraction'stocfirst, then a
port of thepdffigures2SectionTitleExtractor rule set over
BlockStyle(font-size median, bold-majority, above-gap ratio,
numbered prefixes including the CJK\u{7b2c}…\u{7ae0}/
\u{7b2c}…\u{8282}families). Cross-page running headers whose
case-folded text recurs on three or more pages are dropped from the
candidate pool before promotion. The selected signal is recorded on
Provenance::source_of_structureasoutline/heuristic/
mixed/none.glean_paperruns the pass against PDF
extractions inside its EXTRACT stage; book-side adapters are not
affected. A newBlockKind::Abstractvariant is reserved for the
paper structuring pass — book-side adapters never emit it. -
bookrack doctor --install-pdfium: downloads the pinned PDFium
build, verifies its SHA-256 against the pin, and unpacks the
library into a per-user managed directory that the loader searches
automatically. The first-run wizard offers the same download when
the library is missing. -
metadata.set/metadata.clearaccept areasonthat lands on the
audit row, closing the gap where field edits — unlike approve / ack /
reject — could never record their justification. The CLI and REPL
expose it as an optional--reasonflag; the MCP write tools require
it, so an LLM edit always carries its rationale in the audit trail. -
show_book(and themetadata.showreport that embeds it) lists the
book's active overrides — which fields are curated rather than
extracted, by whom and when, with notes — alongside the merged
effective_biblio, so curation state is visible without replaying
the audit trail. -
metadata.setaccepts aconfirmedmark (CLI/REPL--confirmed,
MCP optionalconfirmed: true) recording that the curator checked
the value against the source itself, not an external catalog. The
audit grades a confirmed override strong unless a validation check
(ISBN checksum, BCP-47 syntax, year range, language/body mismatch)
fails. Rewriting the field without the mark drops it. -
metadata.void(CLI/REPLbookrack metadata void, MCP
library.metadata.void): suppress a field's extracted value with a
NULL override when the value is known wrong and no correct value is
at hand — the field reads as absent until one is set.metadata.clear
removes the suppression. The MCP tool requires areason. -
metadata.reaudit(CLI/REPLbookrack metadata reaudit, MCP
library.metadata.reaudit): re-run the metadata plausibility audit
from the book's cached extraction envelope — no re-extraction — so
the stored verdict / confidence reflect the current effective
metadata after field corrections instead of reporting the
ingest-time outcome forever. Only the rollup pair and one
pipeline-audit row are written; the review status is untouched.
Returns the previous and new verdict / confidence. -
library.show_metadata_report(MCP): recompute the metadata
plausibility audit from the book's cached extraction and return the
full per-field report — origin (extracted/override/
override_confirmed/voided), grade, flags, and hint per field,
plus TOC shape flags and copyright-page block candidates — next to
the stored rollup and review status, so a curator can see why a
record readsneeds_work/lowwithout re-deriving the signals.
Pure read, runs the default audit profile;metadata.reaudit
remains the write path that refreshes the stored rollup. -
Contributor curation:
metadata.contributor_addwrites an
origin = "user"attribution (closed role set:author/
translator/editor/other) that survives re-ingest and is
immediately matched byfind_bookscontributor_name;
metadata.contributor_removedeletes one row by the
contributor_idnow listed inshow_book, whatever its origin —
the path for stripping a wrong extracted attribution. CLI/REPL:
bookrack metadata contributor-add/contributor-remove; MCP:
library.metadata.contributor_add/_removewith a required
reason.
Changed
-
M8 catalog migration: state tables shed their
book_prefix in
favour of item-shaped names (item_review,item_pipeline_audit,
…) and gain akindcolumn where the table is shared between
books and papers; scope strings in the catalog API are replaced by
core::ItemKind. New paper-shaped columns land onbiblio,
contributor, andintake(source_pdf_path). The migration
runs automatically at daemon preflight against an older library;
the per-database backup it writes is clustered by database stem so
the books-side and papers-side migrations cannot overwrite each
other. -
All MCP library read tools now reach the catalog and corpus through
the control plane, closing the last direct-handle path. The
standalonebookrack-mcpprocess and the in-tree daemon share one
in-process scheduler for both reads and writes — completing the
control-plane split started in 0.4.0. -
The extraction envelope type is hoisted out of
bookrack-ingest
intobookrack-extract, so the book-side ingest pipeline and the
paper-side glean pipeline consume one shared definition. -
extract_paper_structuredis now precision-first: it returns an
empty TOC rather than guess. The pass runs in two cooperating
stages — outline-guided heading promotion and a strict numbered
heuristic with sequence validation — both gated so a noise line
cannot survive: a candidate must carry a numbered prefix
(N,N.M,N.M.P, Roman, orAppendix), be single-line, fit
under 80 characters, have no math / arrow / geometric-shape
Unicode, no@, no boilerplate prefix (we,the,vol.,
https://, …). The sequence check accepts only candidates that
advance the ascending series at their level, and the outline pass
matches anchored entries against block text by label prefix (with
numeric / Roman str...
v0.4.0
Added
- JSON-RPC control plane: the daemon serves a control socket, and
every other process is a client of it. The method surface covers
the write commands with progress events, queue lifecycle
(queue.pause/queue.resume/queue.clear),verify.run,
library.fork,diagnose.run, introspection (daemon.methods,
daemon.mcp_tools), and a subscribable log/event channel. bookrack repl: standalone control-socket client process. Reedline
runs in the client; every command is dispatched as a JSON-RPC call
over the daemon's control plane. The prompt shows a live
[<state-indicator>bookrack:<library>/queue:<n>] >status line
driven byevents.subscribe. When stdin is not a TTY the client
runs a batch loop: each line is parsed via the shared
bookrack-repl-grammarand dispatched in sequence, and the
process exits non-zero on the first RPC failure. When the daemon
is not running the client exits with code 2 and the message
bookrack daemon not running; start it with: bookrack run.library.read_contextandlibrary.read_spanMCP tools:
structural passage reads around a hit, backed by a document-order
leaves_in_doc_spancorpus query, so an agent can widen a citation
to its surroundings without a second search.- Search citations carry the intake id and structural anchors of the
cited span. crates/app: Tauri shell hosting the daemon in-process with a
minimal Svelte 5 panel — groundwork for the tray GUI, not yet part
of the release artifacts. Control-plane DTOs and events derive
ts-rs schemas, generated as.tsfiles for the webview at test
time.WizardDrivertrait behind the first-run wizard, with the CLI
driver as the first implementation and room for a GUI driver.- The daemon raises its soft
RLIMIT_NOFILEto the hard limit at
startup, andbookrack doctorreports the effective limit. A
GUI-launched daemon previously inherited a 256-fd default and
starved mid-batch on LanceDB fragment files.
Changed
bookrack rundefaults to the silent daemon: it no longer reads
stdin, no longer spawns reedline, and emits no banner. Open an
interactive REPL withbookrack replin another process.- One-shot subcommands are rewritten as control-plane clients and
dispatch their work to the running daemon over the control socket. - A second
bookrack runagainst a live session brings the existing
daemon to the front and exits cleanly;bookrack-mcpgains the
same re-entry behaviour. - The ingest queue pauses on process-level failures instead of
rapidly failing every remaining job; the error is the process's,
not the books'. - Ingest error strings keep their flattened source chains, so root
causes such asToo many open filesreach the pipeline audit
instead of an opaquevector store error.
Deprecated
bookrack run --legacy-repl: one-release transition flag (hidden
from--help) that re-enables the in-process reedline REPL for CI
scripts that fed it via stdin. The flag will be removed in the
next release after this one; migrate tobookrack repl(which
accepts stdin in batch mode for the same scripted use case).
Fixed
- The RESET confirmation on destructive vector-store commands
survives the control-plane client split. - The daemon process exits after a control-plane shutdown request
instead of lingering, and the--legacy-replpath prints its
session banner again. - The hint printed after
library.forknames the correct
model-switch command.