Knowledge Base and RAG

Knowledge Base and RAG

ChatHub Knowledge uses real vector retrieval backed by PostgreSQL pgvector and a dedicated external embedding provider. It does not silently reuse the active chat model, a chat-provider API key, or DEFAULT_FILES_CONFIG.

When no complete embedding provider is available, Knowledge shows a warning banner. You can still upload and chunk supported documents, but vector indexing, semantic retrieval, chat RAG, and RAG evaluations do not work until the provider is configured.

A chat RAG query can run inside a background conversation turn using the same dedicated embedding provider. See Background Conversation Generation.

When the turn has to stay in the browser (for example the assistant has code interpreter or an image-generation chat tool installed), Knowledge retrieval still continues after you switch topics. The assistant row keeps its loading state during retrieval so you do not see an empty “Reference Source” bubble. Stop and account switch still cancel it.

Open Knowledge

• On desktop, select Knowledge Base in the left sidebar.
• On mobile, select More in the bottom navigation, then select Knowledge Base.

On a compact screen, Knowledge opens directly to the All Files workspace filtered by the active category. The header shows the active category (or the named Knowledge Base you opened), a menu button on the left, and the upload action on the right. Tap the menu button to open a left navigation drawer with All Files categories and the list of named Knowledge Bases. Selecting a category closes the drawer and shows the home workspace filtered by that category (categories always return you to All Files, even if you opened the drawer from inside a named Knowledge Base). Selecting a named Knowledge Base closes the drawer and opens that base's workspace. Browser or hardware Back returns to the previous workspace because each selection updates the real URL.

The bottom navigation stays visible the whole time — Knowledge no longer hides it to show the file workspace. The compact search field and file rows are touch-friendly: date and size columns collapse under the filename and the chunk drawer opens at full screen width. Direct links to /knowledge/bases/:id or /knowledge?file=<id> open the matching workspace or file preview modal; deep links to /knowledge/bases canonicalize to /knowledge and open the drawer so you can pick a Knowledge Base.

Requirements

• PostgreSQL with the vector extension. The recommended Compose image is pgvector/pgvector:pg16.
• A complete OpenAI, Cohere, or Voyage embedding configuration.
• A stable KEY_VAULTS_SECRET when account-level provider settings are used.

The database stores 1,024-dimensional vectors and uses an HNSW cosine index. Custom models must return exactly 1,024 finite numeric values for every input.

Configure for the deployment

Set all three required variables together. RAG_EMBEDDING_BASE_URL is optional and must be a secret-free HTTP(S) API root. Do not place credentials, query parameters, or fragments in it; authentication belongs in RAG_EMBEDDING_API_KEY.

OpenAI

RAG_EMBEDDING_PROVIDER=openai
RAG_EMBEDDING_MODEL=text-embedding-3-small
RAG_EMBEDDING_API_KEY=sk-...
# RAG_EMBEDDING_BASE_URL=https://api.openai.com/v1

Cohere

RAG_EMBEDDING_PROVIDER=cohere
RAG_EMBEDDING_MODEL=embed-multilingual-v3.0
RAG_EMBEDDING_API_KEY=...
# RAG_EMBEDDING_BASE_URL=https://api.cohere.com

Voyage AI

RAG_EMBEDDING_PROVIDER=voyage
RAG_EMBEDDING_MODEL=voyage-3.5
RAG_EMBEDDING_API_KEY=...
# RAG_EMBEDDING_BASE_URL=https://api.voyageai.com/v1

Recreate or restart the ChatHub container after changing environment variables. A partial configuration is treated as invalid and keeps the warning visible.

Configure for one account

Open Settings -> RAG Provider. Choose a provider, confirm the model and optional endpoint, enter its API key, then use Test connection before saving. The API key is encrypted in the account key vault and is never returned to the browser after save; the password field displays only a saved-key hint.

A complete account configuration takes precedence over deployment variables. Use Use environment configuration to delete the account override and return to the deployment setting. If an account override is incomplete or invalid, ChatHub does not silently fall back to the environment; correct it or clear it. If ChatHub reports that saved credentials cannot be decrypted, restore the original KEY_VAULTS_SECRET before changing RAG settings. Save and clear are blocked in this state so other provider credentials are not overwritten.

What appears in Knowledge

Knowledge is an explicit document collection, not a list of every compatible file in the account. A file appears only after you upload it from the Knowledge workspace, upload it into a named Knowledge Base, or explicitly add it to a Knowledge Base. Files pasted or attached in a chat remain with that conversation and do not appear in Knowledge merely because their format can be parsed.

Knowledge accepts only formats handled by its document loaders:

• PDF, DOCX, PPTX, EPUB, and CSV
• Markdown, HTML, and LaTeX
• plain text and supported text/source-code files

Images, audio, video, legacy DOC, XLS/XLSX spreadsheets, SQLite/database files, unknown binary files, and archives do not appear. This includes database files mislabeled as plain text. ZIP may be selected as an upload container, but ChatHub stores only supported documents extracted from it; the ZIP itself and unsupported entries are discarded from that Knowledge upload.

Deploying a MarkItDown sidecar widens this list considerably, and converts everything to Markdown before chunking.

Converting any document to Markdown

By default each format is parsed by its own loader, and PDF and PPTX documents are split one chunk per page or slide. Pointing ChatHub at a MarkItDown sidecar instead converts every upload to Markdown first, then splits it on Markdown structure, so headings and tables stay intact in the embedded chunks:

MARKITDOWN_SERVICE_URL=http://markitdown:5000

The sidecar is a separate container because MarkItDown is Python and the ChatHub image ships only Node. MarkItDown Sidecar has the files to build it, how to run and update it, its options, and how to add LLM-vision OCR or Azure Document Intelligence.

Once configured, Knowledge also accepts XLSX and XLS (one Markdown table per sheet), Outlook .msg, Jupyter notebooks, ZIP archives, RSS/Atom feeds, images and audio. Legacy DOC, DOCM and database files remain unsupported — MarkItDown has no converter for them either.

MarkItDown becomes the preferred converter for every format, and the built-in loaders remain the fallback: if the sidecar is unreachable, or has no converter for a format, parsing continues as it did before. Route individual formats explicitly with FILE_TYPE_CHUNKING_RULES, which matches on either the file extension or the MIME subtype:

# keep spreadsheets on MarkItDown, but parse PDFs with Unstructured
FILE_TYPE_CHUNKING_RULES="xlsx=markitdown;pdf=unstructured,default"

Existing documents keep the chunks they were parsed with. Reindex a document to re-chunk it through MarkItDown. In particular, re-parse an older document if a stored table displays literal pipes or has a separator row detached from its header: the current Markdown splitting path repairs only positively identified merged separators, keeps small tables intact, and splits oversized tables into valid Markdown pages no longer than the normal 1,000-character chunk target. Each large-table page repeats its header and delimiter; an individual giant cell uses bounded, column-labeled continuation content without prose overlap. These rules apply when the file is parsed again and cannot repair already-stored chunks.

All Files shows documents uploaded through Knowledge and supported documents already associated with a Knowledge Base. Older documents that were manually parsed through Knowledge before explicit upload tracking was added also remain visible. Turn off Show content in Knowledge Base to hide every KB-member file from the combined All Files list, regardless of its format. This does not delete the files or remove their Knowledge Base associations: each file remains visible and manageable inside the Knowledge Base to which it belongs. Other account files that were uploaded only for chat or another workflow are excluded from All Files.

This filtering does not delete unrelated media. A screenshot sent in a chat topic remains attached to that topic. Images created in the Image workspace remain available from Image and the durable Artifacts gallery.

Indexing and reindexing

Parsing and embedding are separate tasks. A document is ready for RAG only when every current chunk has a vector from the active provider identity and the embedding task completed successfully. Old vectors from another provider, endpoint, or model are not searched.

Successful parsing starts embedding automatically by default. There is no per-document UI toggle; a deployment can set CHUNKS_AUTO_EMBEDDING=0 to stop the automatic handoff and use the explicit reindex action later.

Changing provider, endpoint, or model prompts you to Reindex all documents when existing vectors are present. Rotating only the API key does not change vector identity and does not require reindexing. Returning from an account override to a different environment provider can also require reindexing.

Re-parsing a document atomically replaces its previous chunks and vectors. Retrieval considers the nearest candidates, applies a minimum similarity, and returns a bounded set of relevant chunks to chat. If reindexing is started more than once for the same file, only the newest task is allowed to keep writing.

Reviewing retrieved chunks and source documents

When an assistant message cites retrieved Knowledge chunks, expand the Reference Chunks card under the message and click any chunk to open the File Preview popup. The popup has two tabs when opened from a chat citation:

• Chunk — paginates the chunks retrieved for that message, one chunk per page. It opens on the chunk you clicked; use the numbered pager (or the first/last jumpers) to move between the other retrieved chunks without loading the whole document into one scrollable panel.
• File — paginates every chunk of the source file, one chunk per page, starting from the first chunk. PDF and image files keep their native renderers (PDF.js paginates by PDF page; images render directly).

Clicking any filename or masonry card in the Knowledge file manager opens the chunk drawer, regardless of extension. HTML, text, Markdown, Office, and other Knowledge files therefore share the same local one-chunk-per-page Markdown viewer instead of opening a format-specific fullscreen preview. The drawer's compact Basic Information section stays at the top above search, chunk pages, and similarity results. It shows file size and format, created and updated times, chunk count, embedding status, and a download action.

The drawer's provenance header and detail action follow the selected page. A file with no chunks shows a Parse action only when the current deployment can parse that file. DOCX/PPTX are supported by the built-in loaders, XLS/XLSX are supported when the MarkItDown sidecar is configured, and unsupported legacy Office formats or sidecar-dependent spreadsheets without the sidecar show an explanation without an action. The drawer refreshes after parsing finishes rather than opening a blank or unsupported external preview.

The drawer requests all chunk text in one query for numbered navigation. It returns only the provenance fields used by the header and detail dialog: converted_by, source_file_type, and source_title; converter geometry and duplicated HTML are not sent to the browser.

How Knowledge is used in chat

Knowledge retrieval and query rewriting are automatic when the active assistant has Knowledge enabled. ChatHub searches by cosine similarity (up to 24 nearest candidates, minimum similarity 0.2, at most 8 selected chunks) and does not use a reranker in this path.

Selected chunks are wrapped with the Knowledge instructions and metadata only for the active initial model request. This does consume that request's context window, but the block is not saved or consolidated into chat history and is not carried into later tool continuations. The regular token popover shows the whole generated block as Knowledge Base while it is active; ChatHub does not run an extra retrieval just to estimate it. Token counting is best-effort and never blocks the answer: ChatHub uses exact browser-worker counting when available, then an estimated count, and finally character length if both tokenizer paths fail.

If you arm Context Export Next Request, a RAG request includes the Knowledge Base token allocation and a bounded JSON summary with query-rewrite state, scope/candidate/result counts, similarity threshold and selected scores, plus a diagnostic ID when enabled. countMode identifies the allocation as exact, estimated, or character. The sanitized Engineered Context contains the actual injected block. Group-chat orchestration does not currently add this RAG bucket.

Knowledge Base diagnostics

Set the server/container variable below and recreate the service:

CHATHUB_KNOWLEDGE_DEBUG=1
# CHATHUB_KNOWLEDGE_DEBUG=verbose

The safe level writes structured JSON for upload registration and repair, Knowledge Base association, chunking, embedding/reindexing, retrieval, and prompt injection. verbose adds bounded deployment-keyed fingerprints and shapes. Neither mode records raw file names, document/chunk/query/prompt text, URLs, credentials, private IDs, arbitrary error messages, bodies, or stacks. Relevant UI/task failures show an opaque kb_... diagnostic ID; search the application logs for that exact ID. See Debug and Diagnostics for the event and privacy contract.

For chat requests, prompt_injection_reported includes the token countMode. If preparation fails, client_preparation_failed includes a categorical failurePhase: retrieval, prompt_assembly, token_accounting, or message_metadata.

Troubleshooting

A Knowledge chat stays at ..., especially on iPhone or an installed PWA

Current ChatHub versions fall back automatically when the browser tokenizer worker is unavailable, so a tokenizer failure must not stop the model request. Any other client preparation failure replaces the assistant placeholder with a persistent error. Reload the conversation and copy its kb_... diagnostic ID, then search the safe Knowledge logs.

For a healthy request, the correlated records proceed from retrieval through prompt_injection_reported, followed by the normal provider response and assistant finalization. If retrieval_settled is followed by client_preparation_failed, use failurePhase to locate the browser-side boundary. Raw provider completion debug switches are not needed for this check and may expose prompts or responses.

The Knowledge warning remains visible

Open Settings -> RAG Provider and inspect the status. Configure all required fields, or fix all three required environment values and restart the container. An invalid account override must be corrected or cleared even when the server environment is valid.

Test connection fails

Verify the API key, model ID, and base URL. A base URL should be the provider or compatible API root; ChatHub adds the embedding path. It must not contain URL credentials, query parameters, or fragments. Custom endpoints must use Bearer authentication and return the supported provider response shape with 1,024-dimensional vectors.

A document has chunks but is not retrieved

Check its embedding task status. Reindex after changing provider identity. ChatHub deliberately excludes partial, failed, and stale-fingerprint vectors. Low-similarity chunks can also be omitted from a result.

A PDF finishes with NoChunkError

Typical causes are an image-only scan without an OCR text layer, encryption or password protection, malformed/truncated data, an empty document, or PDF structures the current loader cannot extract. Run OCR on scans, decrypt the file, or re-export it as a clean PDF and upload again. This differs from NoSuchKey/“source file is missing,” which means object storage cannot find the uploaded bytes and is handled by the recovery flow below.

A MarkItDown sidecar can OCR scans for you — see MarkItDown Sidecar → Scanned documents. Uploading an image or scan without OCR configured produces only EXIF metadata, which also reports as NoChunkError.

A document reports that its source file is missing

The database record and Knowledge Base association remain visible so the failure can be recovered. Upload the same file again. ChatHub detects that the hash record points to a missing object, uploads a fresh copy, and repairs the stored reference before parsing resumes. If the document was removed by an older ChatHub version, upload it again and add it back to the Knowledge Base.

Migration reports that vector is unavailable

The PostgreSQL server does not have pgvector installed. Use pgvector/pgvector:pg16 or enable the extension on the managed database, then restart ChatHub so normal migrations retry. Do not manually mark the failed migration as complete.

A picture or spreadsheet disappeared from Knowledge

Expected without a MarkItDown sidecar: Knowledge is a document-chunking surface, and its loaders cannot read those formats. The original file is not deleted merely because it is filtered from Knowledge. Topic screenshots remain in their topic; generated images remain in Image and Artifacts.

Set MARKITDOWN_SERVICE_URL to ingest spreadsheets and images too.