Repository navigation
Replies: 1 comment
|
Carrying forward what was added on #6522 after the original text, since the comments stayed on the closed issue:
Why we still think storage matters: each of these results needed the synopses generated again for that one experiment. With stored synopses (keyed by chunk text hash, synopsis model and prompt version, as in the original text) a brain could audit them, re-score writers, and upgrade page by page instead of regenerating every chunk (about 145k for us). Where we are: we stopped generating new synopses on 28 Oct and keep the existing vectors. We would revisit that if synopses were stored. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Moved from #6522, opened by @knicolov on 2026-10-12. Feature requests now live in Discussions > Ideas.
What problem does this solve?
In
per_chunk_synopsismode the synopsis is generated, wrapped into the embedded text, and then discarded.contextual-retrieval-service.tscollects synopses and embeddings in memory in phase 1; phase 2 writes only the chunk vectors and the mode stamp, andchunk_textstays canonical. The LRU inpage-summary.tsis process-local, and its key includescorpus_generation, which hashes the embedding model tag.So on a lifted brain, any change of embedding model or dimensions re-runs one LLM call per chunk. On our brain that is about 145k chunks, which on subscription limits means days of calls. The synopsis text can also never be read or audited after the fact.
This is separate from #6341, prompt caching for synopsis calls (in v0.60.177.0). Caching lowers the cost of generating a synopsis, but the text is still discarded after embedding.
We measured what the synopsis is worth, which is why losing it matters. On 29 gold questions over a 23k-chunk slice (dense retrieval, recall@10 of answer pages), bge-m3 reached 0.670 on raw chunk text and 0.834 on gbrain's production vectors with the title + synopsis context. Another embedder reached 0.811 on raw text, but switching to it would mean regenerating every synopsis first.
What does the solution look like?
Persist the synopsis when it is generated, keyed independently of the embedding model. For example, a table with
page_id,chunk_index,chunk_text_hash,synopsis,synopsis_model,prompt_versionandcreated_at. Reindex writes it, and re-embed reads it whenchunk_text_hashandprompt_versionmatch, whatever the embedding model. A read command such asgbrain synopsis show <slug>would make synopses auditable.Alternatives considered
All reactions