RAG and Knowledge

RAG and Knowledge

Layers

flowchart TB
  Refs[Project references] --> Ingest[chunk + embed]
  Ingest --> PC[Chroma: anylm_project_context]
  Chat[User message] --> EmbedQ[embed query]
  EmbedQ --> PC
  PC --> Prompt[System prompt chunks + sources]
  Threads[Prior turns] --> Mem[anylm_project_memory]
  Mem --> Prompt
  Org[General/org KB] --> VS[vectorstore collections]
  VS --> Prompt
  Turns[Completed turns] --> Remember[memory + graph extract]
  Remember --> Mem
  Remember --> KG[Local knowledge graph]

Loading

Modules

Module	Responsibility
rag.ts	Chunking, similarity, top-k helpers
context.ts	Ingest references; retrieve for prompt
embed.ts	Ollama embeddings
chroma.ts / chroma-server.ts	Client + bundled server lifecycle
memory.ts	Cross-thread recall / remember
vectorstore.ts	General / org knowledge
graph/	Entity/relation store + extract

Embeddings

• Default model: nomic-embed-text (override ANYLM_EMBED_MODEL).
• If embeddings unavailable, chat falls back to stored summaries for references.

Chroma soft-fail

Startup tries to ensure Chroma (startup-deps.ts). If the server or collections fail:

• Reads return empty
• Writes may be dropped
• Chat still works with weaker context

Never hard-crash the app on vector DB errors when adding features.

Ingest path

Adding a reference (text-like: .txt .md .json .csv .log, plus richer parsers where implemented):

1. Read file contents in main
2. Chunk
3. Embed chunks
4. Upsert into project collection (user-scoped)
5. Store short summary for UI / fallback

Retrieve path

On chat:

1. Embed latest user text
2. Query top-k chunks with source names
3. Inject into system prompt
4. Optionally blend memory / org hits

Knowledge graph

Separate from vectors: structured entities/relations extracted around turns (graph/). Useful for “what do we know” style grounding; inspect store module when extending.

Workspace vs RAG

• RAG references are curated project docs.
• Workspace FS tools operate on a picked folder for coding — different subsystem (workspace.ts, project-coding/).

Contributor tips

• Keep chunk sizes / top-k coherent with context window of local models.
• User-scope collections carefully (multi-account machines).
• Add tests for thin/invalid document rejection (see documents tests).
• When changing collection names, plan migration — silent empty RAG is a common footgun.