How are people setting up a persistent knowledge base / document Q&A?

[OPhineas]

I'm using Odysseus (Docker, Windows, local model via Ollama qwen2.5:7b) to build an internal assistant that answers staff questions from a set of company SOPs. Loving the chat + local model + privacy setup. Where I'm stuck is getting documents in persistently so they're retrieved automatically in normal chat.
What I've found so far:

Attaching a file in chat works, but only for that single message, the next question doesn't have the document, and MANAGE_DOCUMENTS → list returns nothing.
Brain → Memories import shreds a document into individual facts, which isn't the right shape for multi-page SOPs.
Library → Create → Import from file saves a document (PyMuPDF installed for PDFs), but I can't tell if/how the chat or agent actually retrieves from it.
In Agent mode, MANAGE_DOCUMENTS searches but finds nothing unless the doc is in the right place and I'm not sure where that is.

Library → Import (the Import button directly) always reports "imported 0 files" no error, no file added. Watching the browser Network tab, no POST request fires when I select a file, so the file never leaves the browser. Separately, importing via Brain → Add failed with session not found - needed for llm config until I started a fresh chat and sent a message first — after that, Brain import worked. So the import paths seem to depend on an active session, and the Library Import button in particular never sends anything.

Question: What's the intended flow for "load documents once → ask questions and have the relevant passages retrieved automatically"? Is there a RAG/vector pipeline over the Library I'm missing, or is the current path mainly per-message attachment + Memories? Anyone running a working SOP/knowledge-base setup. How did you wire it?
Setup: Docker Compose (Odysseus + ChromaDB + SearXNG + ntfy), ChromaDB healthy, MemoryVectorStore initialized in logs.

[tkwr441]

also struggling here. the memory management is odd. it worked once but i cant get the memories to be invoked so it hallucinates often.

[christophermilian]

I'm actively working on upgrading document processing so that:

1. Attachments up to 50MB (currently capped at 10MB):

• "≤10MB" → handled in memory (current inline behavior, unchanged)
• ">10MB" → processed as an async background task that chunks + embeds the doc into RAG, so it persists and is retrieved automatically in later messages (not just the one message it was attached to, the core thing this thread is hitting)

2. An agent tool that returns document-processing status, so the assistant knows when a large doc is still indexing and can wait / tell you, then answer once it's ready.

Happy to share a proposal issue once I've written it up.

[CoolJohn-lab]

50MB document would nuke your context window from orbit though, even if you have hundreds of gigs of ram. What document is even 50MB?

[christophermilian]

After the document is processed in permanent storage, the agent would find the relevant parts only. Pull info as needed vs loading the entire document at once.

[christophermilian]

In my use case I have a 37MB PDF for an AWS course that I wanted the project to help me study. Using main it totally poops out silently, leaving the user confused.

[CoolJohn-lab]

You should read up on Recursive Language Models if you really want to reason over huge documents.

[christophermilian]

@OPhineas

Is this the feature flow you want? This is something I got working just now.

[OPhineas]

Yeah, that's probably it. I just wanted a doc that stays indexed and gets retrieved automatically in later messages, not just the one it's attached to. My goal is really just loading a set of SOPs once into a persistent knowledge base that every chat retrieves from not per-conversation attachments. So, the key questions for me are:
Does this work for documents loaded once and shared across all chats/users, or is it scoped to the conversation you attach them to?
Is the indexed content shared, or per-session?
If it's a shared, load-once knowledge base, that's exactly what I need... thanks for looking into this

[christophermilian]

Yup I also have cross-chat file uploads visible!

[Ber-Vazq]

@christophermilian How'd you get this to work? I was wanting to upload some textbooks and ttrpg sourcebooks

[CoolJohn-lab]

Sounds like you want a new feature to be able to permanently pin specific document(s) to a chat, and not have to keep re-uploading it for each new chat. I would argue that uploading the document once and having the LLM create separate memories for each instruction in the SOP, and another for what order each step needs to be in is a better way of doing that. With that said I can understand wanting a simpler way to do it, anchored to a static file, that you can upload a new version of as needed, without having to create new memories each time.

[CoolJohn-lab]

Check out Recursive Language Models as a neat fix to this issue. You don’t load the document into context, you load it into a huge Python variable and let the model use Python code to read only the parts it needs, or call sub agents to read the parts it needs. Seems to fit your use case well.