Gather books onto a shelf and ask the shelf

[bokuweb]

A question could only be put to a passage the reader had already marked, which
makes a long book worse to ask about than to read, and makes a question that
spans two books impossible to ask at all.

A shelf is books gathered so they can be asked together. Make one with the
folder button in the sidebar header, drag books onto it or file them by
right-click, and click it to open: the books on it in the middle, the
conversation with it on the right. Nothing is excerpted, because there is no one
book to excerpt — searching every book on the shelf for the question is what
produces the context.

Both kinds of question run the same course — same turns, same streaming, same
citations — and differ only in what context is gathered and where the reader
finds the conversation again. That difference is named once, as Conversation
and Subject, rather than assumed in a dozen places.

Decisions

• Shelves are flat. A question put to a tree would have to say how deep it
  goes: a thing to explain and a thing to get wrong, in return for an
  arrangement a library this size does not need.
• Deleting a shelf keeps its books. An arrangement of the library is not
  part of it. The conversation does go, because it was the shelf's.
• The model is never asked which book it is quoting. A source is resolved by
  looking its quotation up in each book until one holds it. A title copied
  slightly wrong would cost the reader the jump; a quotation copied slightly
  wrong is already handled by the fragment matching the citation lookup has
  always done.

The migration

A conversation belonged to a highlight, so chat_messages.highlight_id was
NOT NULL. SQLite will not drop that, so the table is rebuilt and its rows
carried across in one transaction. A reader's conversations cannot be rebuilt
from their PDFs, so this has a test of its own and was run against a copy of a
real library before being run against the real one: 21 messages, 14 highlights,
integrity_check and foreign_key_check clean.

A filter taken back

The second commit removes the keyword filter that landed with retrieval in #4,
because measuring it on real questions showed it was worse than nothing.

It asked how much of the question a passage contained, which fails by
length — a passage that answers a forty-token question still holds a twentieth
of it, so a threshold that is right for a phrase throws away every answer to a
sentence. Asking 「runtime が edge で動くという話はどの本?」 dropped the one book
that says exactly that, in those words.

The replacement tried next, matching on the rarest half of the question's words,
fails by subject: in a book about primes, 素数 is one of the commonest words
in the index, so "rarest half" discards the word the question is about. It
ranked a page of page numbers above the chapter on generating primes.

Both treat a term's weight as something a caller can recover after ranking. It
is not — bm25 already knows every term's weight and has spent it. The fix
belongs where the ranking is: capping what low-information terms contribute, or
segmenting Japanese into words rather than pairs. Until then retrieval is
ungated, because missing the passage that answers the question is a worse
failure than carrying one that does not. The vector floor stays; it is a
measured property of the model rather than a guess about words.

Verified

Against a copy of a real library, with the installed claude:

cargo run -p pedro-core --example ask_shelf -- "素数の生成について、どの本のどのあたりに書いてある?"

Eight citations, every one resolved to a page and a book, conversation filed
under the shelf. 267 tests pass across the workspace.

Not verified: I could not click the UI — this machine has not granted
Accessibility to automation — so the shelf screens are compile-checked and
reasoned about, not driven. The sidebar was seen in a screenshot; the shelf view
and the drag-and-drop were not.

Rebased onto main

This was branched before #2, #3 and #4 merged. Rebasing conflicted in four
places, all in pedro-app where the Drive work added to the same spots:

• app.rs — Drive's fields and the shelf's; both kept.
• reader.rs — one import line, Spinner against Input; both kept.
• sidebar.rs header — the Drive button and the new-shelf button; both kept.
• sidebar.rs — git spliced render_drive_button/render_drive_field and
  render_new_shelf_button into one function, because they share the shape
  if … { return None; } Some(. Taking either side would have dropped a
  feature, so the three functions were written out again.

🤖 Generated with Claude Code