Knowbase — Multi-Agent Knowledge Workspace

Note: "Knowbase" is this project's working name, chosen for this portfolio piece. It is not affiliated with any other product or company that may share a similar name.

Production-minded multi-agent knowledge workspace demonstrating RAG, evaluation, MCP, workspace isolation and observable AI workflows. Upload documents, ask questions, and get answers grounded in your own knowledge base — with inline citations down to the chapter level.

Answer format is content-aware, not one-size-fits-all. The synthesis prompt (backend/app/agents/nodes/synthesize.py) detects two question shapes at answer time: ordinary questions get a standard grounded Q&A answer (the general case); questions that look like a structured academic assignment (multiple numbered sub-questions, point values, task verbs like "Diskutieren"/"Analysieren", requests for a personal example) get a distinct structured format instead — German-language academic-assignment phrasing throughout, since this behavior was originally built around the author's own distance-learning coursework. This is a genuine, deliberate feature (a worked example of prompt-level intent routing within a single LLM call, not two separate code paths) rather than a generic-positioning bug, but it's real product behavior worth knowing about before evaluating this as a general-purpose team tool: an unrelated multi-part business question can trigger the assignment format's German scaffolding by accident if it happens to match those signals.

Browser
  │  SSE token stream        REST + multipart upload
  ▼                          ▼
Nginx ──────────────────── Next.js Frontend (port 3030)
  │                          │ typed API calls
  ▼                          ▼
FastAPI Backend (port 8000) ── PostgreSQL + pgvector
  │ asyncio.create_task        │
  ▼                          ▼
LangGraph Agent            Redis
  ├─ retrieval_node  ─────────├─ SSE stream buffers (sse:run:{id})
  ├─ web_search_node           ├─ ARQ job queue
  ├─ memory_read_node          └─ rate limit counters
  ├─ synthesize_node
  └─ memory_write_node       ARQ Worker
                               └─ ingest_document_task
                                    download → extract → chunk → embed → store

Problem

Teams build knowledge in documents, Notion pages, and PDFs — but can't search across them naturally. Consumer AI products (ChatGPT Projects, Claude Projects) now offer their own document/knowledge features too, so the gap isn't "no access to private knowledge" anymore — it's that those are closed, hosted products with no visibility into how retrieval, citation, or access control actually work underneath.

Teams need more than document chat: they need an inspectable and self-hostable workflow with explicit workspace isolation, controlled ingestion, measurable retrieval quality, citations, MCP access, and operational observability. Knowbase demonstrates how these concerns can be implemented together in one repository — not a claim that no other open-source option does, but a worked example of the full pipeline (vector DB, embedding, queue, storage, auth) in a single repo instead of stitched together from five separate services.

Approach

Knowbase solves this with a stateful multi-agent pipeline built on LangGraph:

Ingest — documents are uploaded, extracted (PDF/HTML/TXT), chunked at sentence boundaries with overlap, embedded via OpenAI, and stored in PostgreSQL + pgvector. This runs asynchronously via an ARQ worker so uploads return instantly.
Retrieve — on each user message, a retrieval node and a memory-read node run in parallel. The retrieval node does cosine k-NN search over the user's workspace chunks; the memory node fetches facts the agent has written in previous sessions.
Synthesize — Claude (or GPT-4o-mini) generates a grounded answer that cites the exact source chunks. Tokens stream to the browser in real time via Server-Sent Events over a Redis stream buffer.
Remember — after each answer, a lightweight Haiku pass extracts new facts and writes them back as workspace memories, so the agent improves with every conversation.

Every agent run is recorded in agent_runs (status, timing, token usage). Tool calls and agent progress events (execution trace) are streamed live to the browser as SSE events, not currently persisted to the database — see Known Limitations.

Run locally

cp .env.example .env  # add OPENAI_API_KEY
docker compose up
# → http://localhost:3030

Documents are stored on the local filesystem by default — no separate storage service needed. Seed data is opt-in via SEED_DEMO_DATA=true in .env (see Quick Start below).

Features

AI Chat — Streaming responses token-by-token via SSE, powered by Claude or GPT-4o-mini
Strict Grounding — Answers cite exact source chunks from your uploaded documents; uncovered topics are explicitly marked rather than hallucinated
Web Search Fallback — When your documents have no matching passage, Tavily web search runs automatically and results are clearly labelled as web sources
Document Pipeline — Upload PDF, TXT, MD, HTML, JSON, Excel (.xlsx only — legacy .xls is rejected), Word (docx), PowerPoint (pptx); automatic chunking + vector embedding via ARQ worker
Inline Citations — Every paragraph cites the source document and chapter verbatim (*(Quelle: Document, Chapter)*)
Consistency Check — Compare "subject" documents (e.g. interview transcripts) against a workspace's "global context" documents (e.g. company strategy) in a dedicated compare mode. Retrieves each subject document individually so none get crowded out, then produces a Markdown table verdicting each one (aligned / contradicts / not addressed) with citations.
Memory — Agent writes and reads facts across conversations (workspace + global scope)
MCP Server — Standalone FastMCP server exposes the workspace knowledge base to any MCP-compatible AI client (Claude Desktop, Cursor, etc.) via three tools: search_knowledge, list_documents, get_document
Personal Access Tokens — Machine-to-machine auth for MCP and scripts: create kb_<hex> tokens scoped to a workspace; SHA-256 hashed at rest, shown once on creation
Drag & Drop Upload — Two separately bordered, separately drag-and-droppable areas on the documents page: "Globaler Kontext" (context documents for the Consistency Check) and regular documents
Dark Mode — Toggle in the sidebar, persists to localStorage
Workspace Members — Invite collaborators by email, manage roles (owner / editor / viewer)
Conversation Management — Rename, delete, auto-title on first message
Observability — Prometheus /metrics, structured JSON logs with request IDs
Eval Harness — Golden-dataset evaluation suite: citation accuracy (deterministic, runs on every PR), retrieval recall (embedding-based, nightly CI), groundedness + LLM-as-judge (manual trigger, real LLM calls)

Modules

Each workspace has five modules accessible from the sidebar:

Chat

The main interface. Users type a message and the agent responds in real time — tokens stream word-by-word via SSE. Each answer cites the exact source chunks it was grounded on. The agent remembers facts from previous conversations within the same workspace. Multiple conversations per workspace are supported — create new ones and switch between them via the tab bar; each is persisted independently.

A 🔍 Konsistenz-Check toggle next to the send button switches the next question into compare mode: instead of a normal grounded answer, the agent retrieves every non-context ("subject") document in the workspace individually and produces a Markdown table comparing each one against the workspace's global-context documents, with a verdict (Stimmt überein / Widerspricht / Nicht behandelt) and citations per row.

Documents

Upload and manage the workspace knowledge base. Supported formats: PDF, TXT, Markdown, CSV, HTML, JSON, Excel (.xlsx only — legacy .xls is rejected), Word (docx), PowerPoint (pptx). Each file goes through an async pipeline: extract text → split into chunks → generate vector embeddings → store in pgvector. Status badges (processing / ready / failed) update automatically. Failed documents can be retried without re-uploading.

The page has two separately framed upload areas. 🌐 Globaler Kontext is the authoritative source for the Consistency Check (e.g. a company strategy deck) — anything uploaded here is excluded from normal per-document comparison and instead treated as ground truth. Everything else uploaded to the regular Dokumente area is a "subject" document. Each area has its own upload button and its own drag-and-drop zone.

Tasks

A Kanban board for workspace-level tasks. Four columns: Open → In Progress → Done → Cancelled. Tasks can be created inline, dragged between columns, and optionally tagged with an assigned_agent label. Animated with Framer Motion. Note: assigned_agent is a plain metadata field today — nothing reads it to actually trigger agent execution of the task; see Roadmap.

Members

Invite collaborators to the workspace by email, at any role including Owner — three roles: Owner (full control), Editor (read + write), Viewer (read-only). Any member, including an owner, can be removed by an owner, as long as at least one owner remains afterward — see Roles & Permissions below for exactly what each role can do.

Memories

The agent's persistent fact store. After each conversation the agent automatically extracts key facts and saves them as workspace-scoped memories. The POST /memories API additionally supports user-scoped and global-scoped memories (see Roles & Permissions for how visibility differs by scope). Memories are embedded and used as additional context in future answers — the longer a workspace is used, the more precise the agent becomes.

Roles & Permissions

Every workspace member has exactly one role: viewer, editor, or owner (ranked in that order — each role includes everything the one below it can do). Enforcement lives in backend/app/core/authz.py and is applied per-request on the backend; the frontend additionally hides controls a role can't use, but the backend check is the actual security boundary.

Action	Viewer	Editor	Owner
View documents, tasks, conversations, memories	✓	✓	✓
Upload / delete documents, re-trigger ingestion		✓	✓
Create / edit / delete tasks		✓	✓
Create / rename / delete conversations, send messages		✓	✓
Create workspace-scoped memories		✓	✓
Add / invite members, set their role			✓
Remove members			✓

A workspace always retains at least one owner — removing the last remaining owner (including removing yourself) is rejected, so a workspace can never end up without anyone able to manage it.

Memory visibility (scope field on a memory): user-scoped and global-scoped memories follow the user across every workspace they're a member of; workspace-scoped memories are confined to the workspace they were written in. There is currently no sharing of memories between different users at any scope — global here means "not confined to one workspace," not "visible to other people."

MCP Server

Knowbase ships a standalone Model Context Protocol server (mcp-server/) that exposes the workspace knowledge base to any MCP-compatible AI client — Claude Desktop, Cursor, Windsurf, custom agents, etc.

Three tools:

Tool	Description
`search_knowledge(query, k=8)`	Semantic search over workspace documents. Returns top-k chunks with source labels.
`list_documents()`	List all indexed documents in the workspace.
`get_document(document_id)`	Metadata for a specific document.

Setup:

# 1. Create an API key in the Knowbase UI → Settings → API Keys
#    (scoped to the workspace you want the MCP server to access —
#    the workspace is fixed by the key itself, there is no separate
#    workspace ID setting)

# 2. Configure the MCP server
cd mcp-server
cat > .env <<'EOF'
KNOWBASE_API_URL=http://localhost:8000
KNOWBASE_API_KEY=kb_yourtoken
EOF

# 3. Install and run
uv sync
knowbase-mcp     # starts stdio MCP server

# Or run directly
uv run python -m app.main

Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "knowbase": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/knowbase/mcp-server", "knowbase-mcp"],
      "env": {
        "KNOWBASE_API_URL": "http://localhost:8000",
        "KNOWBASE_API_KEY": "kb_yourtoken"
      }
    }
  }
}

The MCP REST endpoints (POST /mcp/search, GET /mcp/documents) are rate-limited (20 req/min for search, 60/min for document reads) and accept only API key auth — no JWT. The workspace is derived from the key; callers cannot access other workspaces.

Personal Access Tokens (API Keys)

PATs enable machine-to-machine access without going through the browser auth flow.

# Create a key (JWT required — do this in the Knowbase UI or via curl)
curl -X POST http://localhost:8000/api/v1/auth/api-keys \
  -H "Authorization: Bearer <jwt>" \
  -d '{"name": "My MCP key", "workspace_id": "<uuid>"}'
# → {"key": "kb_<64 hex chars>"}   ← shown once, store it securely

# List active keys
curl http://localhost:8000/api/v1/auth/api-keys \
  -H "Authorization: Bearer <jwt>"

# Revoke
curl -X DELETE http://localhost:8000/api/v1/auth/api-keys/<key-id> \
  -H "Authorization: Bearer <jwt>"

Keys are stored as SHA-256 hashes — raw tokens are never persisted. SHA-256 is sufficient here because tokens are 256-bit random (no brute-force risk unlike passwords).

Stack

Layer	Technology	Version
Frontend	Next.js App Router, TypeScript strict	16.x
Styling	Tailwind CSS v4, shadcn/ui primitives	4.x
Server State	TanStack Query	v5
Client State	Zustand	v5
Auth	NextAuth v5 (JWT + refresh tokens)	v5
Backend	FastAPI async	0.136+
Agent	LangGraph stateful graph	1.2+
ORM	SQLAlchemy async + Alembic	2.0
Validation	Pydantic v2	2.x
Vector Search	PostgreSQL + pgvector (HNSW cosine)	`pgvector/pgvector:pg16` image, Python client 0.4+
Queue	Redis + ARQ	—
Storage	Local filesystem (dev) / S3-compatible (prod)	—
Embeddings	OpenAI `text-embedding-3-small`	—
LLM	Anthropic Claude (Sonnet 4.6 default, Haiku 4.5 fallback) or OpenAI GPT-4o	—
MCP Server	FastMCP (stdio transport)	1.9+
Metrics	Prometheus client	—
Logging	structlog JSON	—

Quick Start

# 1. Clone and configure
git clone https://github.com/Bogbra/knowbase.git
cd knowbase
cp .env.example .env
# Edit .env — set OPENAI_API_KEY (required), ANTHROPIC_API_KEY (optional)

# Optional: set SEED_DEMO_DATA=true in .env for a pre-seeded demo login (see below)

# 2. One-command start
docker compose up

# App:   http://localhost:3030
# API:   http://localhost:8000/docs

Documents are stored on the local filesystem by default (backend/dev-storage/, shared between the backend and worker containers via the existing bind-mount) — no separate storage service is required locally. See Storage below for the production S3-compatible alternative.

With SEED_DEMO_DATA=true set in .env, seed data is applied on first run:

Local dev account: admin@knowbase.dev / Admin1234! (local only — never set SEED_DEMO_DATA in a shared/staging/production environment)
Demo workspace: "AI Research 2025" with 2 conversations, 5 memories, 3 tasks

Development (without Docker)

# Terminal 1 — infrastructure
docker compose up postgres redis

# Terminal 2 — backend
cd backend
uv sync
uv run alembic upgrade head
uv run fastapi dev app/main.py

# Terminal 3 — worker
cd backend
uv run arq app.workers.arq_settings.WorkerSettings

# Terminal 4 — frontend
cd frontend
npm install
npm run dev

Environment Variables

Variable	Required	Default	Description
`SECRET_KEY`	✓	—	JWT signing key (`openssl rand -hex 32`)
`DATABASE_URL`	✓	postgres://...	PostgreSQL async URL
`REDIS_URL`	✓	redis://...	Redis connection URL
`OPENAI_API_KEY`	✓	—	For embeddings (`text-embedding-3-small`)
`OPENAI_AGENT_MODEL`	—	`gpt-4o`	OpenAI chat model
`OPENAI_API_BASE`	—	(official)	Custom base URL (e.g. Azure, proxy)
`ANTHROPIC_API_KEY`	—	—	If set, used instead of OpenAI for chat
`AGENT_MODEL`	—	`claude-sonnet-4-6`	Anthropic model
`AGENT_MAX_TOKENS`	—	`4096`	Max tokens per agent response
`AGENT_LLM_TIMEOUT_S`	—	`60`	Timeout (connect + total) for the synthesis LLM call
`WEB_SEARCH_TIMEOUT_S`	—	`15`	Timeout for the Tavily web-search fallback call
`AGENT_JUDGE_MODEL`	—	`claude-haiku-4-5-20251001`	Model used by `eval.runner judge` for groundedness + LLM-as-judge scoring — cheap/fast, since it's called twice per golden question
`S3_ENDPOINT_URL`	—	—	S3-compatible endpoint (omit for AWS S3; set for R2 or another provider). Leave unset for local filesystem storage — see Storage
`S3_ACCESS_KEY_ID`	—	—	S3 credentials (omit for local filesystem fallback)
`S3_SECRET_ACCESS_KEY`	—	—	S3 credentials
`S3_BUCKET_NAME`	—	`knowbase`	S3 bucket
`SENTRY_DSN`	—	—	Reserved for future Sentry integration — not currently initialized anywhere in the app
`ENVIRONMENT`	—	`development`	`development` / `staging` / `production`
`ALLOWED_ORIGINS`	—	`http://localhost:3030`	Comma-separated or JSON-array CORS origins
`AUTH_SECRET`	✓ (frontend)	—	NextAuth v5 signing secret (not the legacy `NEXTAUTH_SECRET` name)
`AUTH_URL`	✓ (frontend)	—	App canonical URL (not the legacy `NEXTAUTH_URL` name)
`NEXT_PUBLIC_API_URL`	✓ (frontend)	—	Backend URL visible to browser — must be passed as a Docker build ARG, not just a runtime env var (see `frontend/Dockerfile`)
`TAVILY_API_KEY`	—	—	Enables web-search fallback when retrieval returns < 3 chunks
`TRUST_PROXY_HEADERS`	—	`false`	Set `true` when behind a trusted reverse proxy (Railway, Fly, nginx)
`SEED_DEMO_DATA`	—	`false`	Opt-in demo seeding (creates `admin@knowbase.dev`). Never set in staging/production

MCP server env vars (set in mcp-server/.env):

Variable	Description
`KNOWBASE_API_URL`	Backend base URL, e.g. `http://localhost:8000`
`KNOWBASE_API_KEY`	`kb_<hex>` personal access token (workspace is fixed by the key itself)

Storage

Two modes, switched automatically based on whether S3_ACCESS_KEY_ID is set (backend/app/core/storage.py):

Local filesystem (default, local dev) — files are written to backend/dev-storage/. In docker-compose.yml, the backend and worker services already share this directory via the same ./backend:/app bind-mount, so no additional setup is needed for uploads to work locally.
S3-compatible object storage (required in production) — set S3_ENDPOINT_URL, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, S3_BUCKET_NAME. This becomes mandatory once backend and worker run as separate, non-bind-mounted containers — which is exactly what docker-compose.prod.yml does (image is immutable, no shared volume between the two services).

No MinIO service is bundled in docker-compose.yml — it isn't needed for local development. If you want to exercise the S3 code path locally, run any S3-compatible service yourself and point the S3_* variables at it.

Deployment

Frontend → Vercel

cd frontend
npx vercel --prod

vercel.json lives in frontend/ (not the repo root) so it's auto-discovered when deploying from that directory. It configures response security headers only — there are no API rewrites. Set Root Directory to frontend in the Vercel dashboard/project settings if importing the repo from its root instead of running vercel from inside frontend/. Set all other environment variables (NEXT_PUBLIC_API_URL, AUTH_URL, AUTH_SECRET, ...) in the Vercel dashboard.

Backend → Railway

railway login
railway link
railway up

The railway.toml at the repo root defines the API service only (startCommand: alembic upgrade head && uvicorn ...). It does not define a worker service — deploying the ARQ worker to Railway requires creating a second Railway service manually in the dashboard, pointed at the same repo/Dockerfile with startCommand overridden to arq app.workers.arq_settings.WorkerSettings. Set all environment variables in the Railway dashboard for both services.

Backend → Fly.io

fly auth login
fly launch --config fly.toml
fly secrets set SECRET_KEY=$(openssl rand -hex 32) DATABASE_URL=... REDIS_URL=...
fly deploy

Migrations run automatically via fly.toml's release_command (alembic upgrade head), executed on a fresh machine before traffic cuts over to the new release — a schema change can't be forgotten on either platform.

Production Docker

cp .env.production.example .env.production
# Fill in all values
docker compose --env-file .env.production -f docker-compose.prod.yml up -d

--env-file is required, not optional: it's what makes Compose substitute ${POSTGRES_PASSWORD}, ${S3_ACCESS_KEY_ID}, etc. inside this YAML file from .env.production. Without it, Compose falls back to a literal .env (or the shell environment) for that substitution — a plain docker compose -f docker-compose.prod.yml up -d silently resolves every one of those to an empty string, with no error, regardless of what's actually in .env.production. A migrate service runs alembic upgrade head once before backend/worker are allowed to start (depends_on: ... condition: service_completed_successfully) — a fresh database has no tables otherwise, since neither service applies migrations on its own.

Nginx listens on port 80 only and handles:

SSE buffering disabled (proxy_buffering off) for the stream endpoint
Rate limiting at the reverse proxy layer
Static file caching for frontend assets

TLS is terminated externally, not by this nginx — e.g. by Fly/Railway's platform-level HTTPS, or an upstream load balancer, matching how the other deploy targets above already get HTTPS automatically. docker-compose.prod.yml intentionally has no 443:443 port mapping or certificate volume mount, and infra/nginx/nginx.conf has no listen 443/HSTS configuration — if you deploy this compose file directly on a host with a public IP, you must put a TLS-terminating proxy in front of it yourself.

Production Checklist

Items marked (enforced) aren't just advice — Settings refuses to boot with ENVIRONMENT=production unless they're satisfied (app/core/config.py).

Development Commands

# Backend
uv run pytest                  # all tests
uv run mypy --strict .         # type check (must be clean)
uv run ruff check .            # lint
uv run ruff format .           # format
uv run alembic upgrade head    # apply migrations
uv run alembic revision --autogenerate -m "description"  # new migration

# Frontend
npm run type-check             # tsc --noEmit
npm run build                  # production build (must pass)
npm run lint                   # eslint + prettier
npm run test                   # Vitest — component/integration tests, all fetch calls mocked
npx playwright install --with-deps chromium && npm run test:e2e  # one real-browser smoke test — needs a running stack, see CI's e2e job

# MCP server
cd mcp-server
uv sync
uv run pytest                  # unit tests (no network)
knowbase-mcp                   # start stdio MCP server

# CI (run all before push)
cd backend && uv run pytest && uv run mypy --strict . && uv run ruff check . && uv run ruff format --check .
cd frontend && npm run type-check && npm run lint && npm run build && npm run test

npm run test (Vitest + React Testing Library) covers component/integration-level behavior with every network call mocked — it is not a browser test. npm run test:e2e (Playwright) is the one real end-to-end test: a real Chromium browser against a real running stack, proving login → session → workspace → page-render actually works together, which mocked tests can't prove.

What CI actually runs (.github/workflows/ci.yml, 7 independent jobs on every push/PR): Backend — Lint / Type / Test (ruff, mypy --strict, pytest), Frontend — Type / Lint / Test / Build, MCP Server — Lint / Type / Test, E2E — Playwright smoke test (isolated docker-compose project, real login), Dependency vulnerability audit (pip-audit for both backend and mcp-server, npm audit --omit=dev), Trivy scan — backend image (HIGH/CRITICAL, ignore-unfixed: true), and Gitleaks — secret scan. Nightly/manual-only workflows (nightly-eval.yml, judge-eval.yml) are separate — see Eval Harness below.

Eval Harness

A golden-dataset evaluation suite lives in backend/eval/. It has three metric levels:

Metric	When	Cost
Citation accuracy	Every PR (unit test)	Free — deterministic regex
Retrieval recall@8	Nightly CI	OpenAI embedding call per question
Groundedness + LLM-as-judge	Manual trigger only	Real, billed LLM calls — generation + 2 judge calls per question

Golden dataset: 37 questions across three synthetic German-language texts covering Marketing-Mix, Transaktionskostentheorie, and Organisationsformen — written from scratch for this eval suite, not sourced from any real publication (see backend/eval/DATA_LICENSE.md). 34 factual/synthesis questions + 3 out-of-domain (expected: no citation).

Citation accuracy and retrieval recall never call the generation LLM — they check citation-label regex and the embedding+vector-search step in isolation. The judge command is the only one that generates a real answer (reusing the actual production prompt) and scores its content: groundedness checks whether the response's factual claims are actually supported by the retrieved chunks, and LLM-as-judge rates overall answer quality/faithfulness 1–5 (with a special rubric for the 3 out-of-domain questions, where correctly declining to answer scores highest).

cd backend

# Ingest corpus into a fresh eval workspace
uv run python -m eval.runner setup

# Run retrieval recall against a workspace
uv run python -m eval.runner recall --workspace-id <uuid> --k 8

# Generate answers + score groundedness/LLM-as-judge (real, billed API calls)
uv run python -m eval.runner judge --workspace-id <uuid> [--limit N]

# Check all metrics against baselines (baselines in eval/baselines/scores.json)
uv run python -m eval.runner check

Baselines are committed as JSON. CI fails when a metric drops below baseline − tolerance. When a score is stable above baseline + tolerance/2, the runner prints a ratchet hint to raise the baseline. judge writes its mean scores into the same scores cache as recall, so check gates on groundedness/judge_score too once baseline entries exist for them — left unset initially, same bootstrapping approach used for recall_at_8.

The nightly workflow (.github/workflows/nightly-eval.yml) runs recall and fails explicitly if OPENAI_API_KEY is not set in CI secrets — skipped tests emit a ::warning:: annotation on main. The judge workflow (.github/workflows/judge-eval.yml) is workflow_dispatch-only — it makes real generation and judging LLM calls, so it isn't run on a recurring schedule; requires both OPENAI_API_KEY and ANTHROPIC_API_KEY as repo secrets.

Architecture Decisions

Repository Pattern (not ActiveRecord)

DB access — query construction and execution — goes through typed repository classes in backend/app/db/repositories/; route handlers and services never write SQL or call session.execute() directly. Services do import SQLAlchemy model classes (e.g. WorkspaceMemberRole) where they need to reference an enum or type, but never bypass the repository layer to query the database themselves. This keeps the data-access layer independently testable and query logic out of services.

LangGraph (not raw LangChain)

LangGraph provides a stateful graph with explicit node transitions, which makes the agent flow easier to reason about and debug than a linear chain. agent_runs.graph_state currently only records final token usage, not a full per-step snapshot — see Known Limitations for what full replayability would need. Raw LangChain chains are harder to introspect and don't support parallel fan-out natively.

SSE (not WebSockets) for Streaming

SSE is unidirectional (server → browser), which is all we need for token streaming. It works over HTTP/1.1, doesn't require a separate protocol handshake, and is trivially proxied by Nginx. WebSockets would add complexity (connection management, heartbeats, reconnect logic) for no benefit here.

pgvector (not Pinecone/Weaviate)

Keeps the stack to one database. The HNSW index on document_chunks.embedding gives sub-millisecond k-NN queries at this scale. For >10M chunks, a dedicated vector DB would make sense, but the operational overhead isn't justified for an MVP.

ARQ (not Celery) for Background Jobs

ARQ is async-native (built on asyncio + Redis), lighter than Celery (no broker/backend config split), and integrates cleanly with the FastAPI async stack. Document ingestion is the only background job type, so Celery's feature set is overkill.

API Key Auth (SHA-256, not bcrypt)

Personal Access Tokens are 256-bit random values (kb_ + 64 hex chars). They are hashed with SHA-256 before storage and looked up by hash. bcrypt is deliberately not used here: its cost factor is designed to slow brute-force attacks against low-entropy passwords. A 256-bit random token has no brute-force surface — the bottleneck is the search space, not the hash speed. Using bcrypt would add ~100 ms of unnecessary latency to every API call without any security benefit.

JWT Rotation Strategy

Access tokens (30 min) are stateless JWTs. Refresh tokens are stored in Redis with their JTI — on each refresh, the old token is deleted and a new one issued. This enables single-use refresh tokens: a stolen refresh token can only be used once before it's invalidated by the legitimate user's next refresh.

PyJWT (not python-jose)

JWTs are signed/verified with HS256 only (settings.ALGORITHM), which needs no ECDSA at all. python-jose[cryptography] pulled in ecdsa unconditionally regardless of algorithm — dead weight that Trivy flagged with a HIGH CVE (CVE-2024-23342, the unfixed-upstream Minerva attack). Migrated app/core/security.py to PyJWT, which needs zero extra crypto dependencies for HS256 (stdlib hmac/hashlib only). uv lock confirmed this removes the entire vulnerable branch — cryptography, ecdsa, pyasn1, rsa, cffi, pycparser — not just ecdsa itself, since nothing else in the project depended on cryptography once python-jose was gone.

Workspace Roles: Viewer / Editor / Owner

A single rank table (viewer < editor < owner in backend/app/core/authz.py, require_workspace_role()) is the one place role sufficiency is decided, reused by every service instead of each one re-implementing its own membership check. Read access (list/get documents, tasks, conversations, memories) only requires membership; content mutations (upload, create, edit, delete) require editor; membership/role management requires owner. The frontend mirrors this with a single useWorkspaceRole() hook so role-gated UI is computed once, not differently per page — the backend check is the actual security boundary regardless, the frontend gate only avoids showing controls that would fail server-side.

Memory Scope Isolation: Three Layers, Not One

The scope/workspace_id pairing on a memory (user/global → no workspace_id; workspace → workspace_id required) is enforced three times, not once: a Pydantic model_validator at the API boundary, a service-layer check in MemoryService.create() for callers that bypass the schema (the agent's own write_memory tool), and a DB CHECK constraint as the final backstop. A workspace-scoped memory write additionally requires editor role in that specific workspace — workspace_id from a request body is never trusted on its own. Without all three layers, a direct-to-service or direct-to-repository caller (present in this codebase, since the agent runtime calls MemoryService directly rather than only through the HTTP API) could still write a cross-workspace memory.

Owner Removal Uses a Row Lock, Not a Count-Then-Delete

Removing a workspace's last owner (or transferring ownership when the account is deleted) can't be a plain "count current owners, then delete if count > 1" — two concurrent removals against a workspace with exactly two owners can each independently observe "2 owners, safe to remove" and both succeed, leaving zero owners. WorkspaceRepository instead issues SELECT id FROM workspace_members WHERE workspace_id = ... AND role = 'owner' FOR UPDATE, locking the actual owner-membership rows (Postgres can't lock rows an aggregate query doesn't return, so SELECT COUNT(*) ... FOR UPDATE doesn't work for this). The count, the sole-owner check, and the delete/transfer all happen inside one transaction with no commit in between; a second concurrent request blocks on the lock and re-reads the already-updated state once the first transaction commits.

Ingestion Idempotency: Delete-Then-Reinsert, Not Insert-Only

ingest_document_task deletes any existing chunks for a document before inserting a fresh set, rather than only ever appending. A retried, manually-requeued, or double-enqueued ingestion run would otherwise leave duplicate or stale chunks behind depending on how the previous attempt ended — a UNIQUE(document_id, chunk_index) DB constraint backstops this even if a future code path forgets to call the delete step first. A mid-loop failure (e.g. chunk 4's embedding call raises after chunks 1–3 succeeded) rolls back the already-flushed chunks before the failed-status commit, so a later retry doesn't add a second, overlapping set on top of leftovers from the failed attempt.

Gitleaks Allowlist: Singular `[allowlist]`, Not `[[allowlists]]`

.gitleaks.toml uses the older, singular [allowlist] table rather than the newer [[allowlists]] array-of-tables form. CI's gitleaks-action pins gitleaks 8.24.3; the plural syntax is silently ignored by that version (config loads without error, but the allowlist is simply never registered), while it works correctly on newer local installs (8.30.1) — a real CI failure this caused and only reproduced by downloading the exact pinned CI binary and testing against it directly. The allowlist also intentionally has no paths entry: a paths filter on a global allowlist unconditionally skips the entire matched file from scanning in this version (confirmed via --log-level debug), which would have suppressed scanning of a real secret anywhere in an allowlisted workflow file, not just the specific known-fake placeholder value.

Known Limitations

Unpatched OS-level CVEs in the base image — the backend/worker image's python:3.12-slim base currently carries several HIGH/CRITICAL CVEs in bundled Debian packages (perl-base, the util-linux family, ncurses, curl) per trivy image scans. These are not introduced by this app's own dependencies — confirmed via docker build --pull (fresh base layer) and an explicit apt-get upgrade -y in the Dockerfile, both showing identical findings, meaning Debian has not yet published patches for these specific CVEs at all, not just that the cached image tag was stale. None of the flagged packages (perl, mount, terminal UI libs) are invoked by the application itself; curl is used only by the container's own HEALTHCHECK. Revisit once upstream patches ship; a base-image migration (distroless/Alpine) would be a larger, separate change given the Dockerfile's current reliance on curl-based healthchecks and Debian-compatible wheels for the Python dependency tree.
No WebSocket support — Agent runs fire-and-forget via asyncio.create_task. If the backend process restarts mid-run, the SSE stream closes. Completed message is still saved to DB.
Single-tenant embeddings — All document chunks in a workspace share the same pgvector table. Cross-workspace isolation is enforced in queries, not at the DB layer.
No file virus scanning, and upload validation is client-supplied MIME type only — the upload endpoint checks the Content-Type header the client sends against an allowlist; it does not additionally check the filename's extension, and does not inspect file contents (no magic-byte/file-signature check) or run ClamAV or equivalent. A file with a spoofed Content-Type (any extension, any actual content) passes this check as long as the header value is on the allowlist.
No pagination on list endpoints — GET /workspaces/{id}/documents, /tasks, /conversations, /members etc. return the full result set in one response. Fine at demo scale; a workspace with thousands of documents would need cursor-based pagination (tracked in Roadmap).
Memories are not shared between different users — scope="global" on a memory means "visible to its owner across every workspace they're in," not "visible to other people." There is currently no mechanism for one user's memory to be surfaced to another user's agent context, at any scope.
Tool calls and per-step graph state are not persisted — tool_calls and agent_runs.graph_state exist in the schema, but the running agent only ever writes graph_state = {"tokens_used": ..., "cost_usd": ...} on completion. Tool-call input/output/duration is streamed live to the browser as SSE events (tool_call/tool_result) and never written to the tool_calls table — AgentRepository.create_tool_call/update_tool_call are defined but unused. A run can currently be watched live, not replayed after the fact.
Token budget is enforced per conversation, checked before each turn — AGENT_TOKEN_BUDGET (default 100k) is a running total across a conversation, read back from the most recent prior turn's agent_runs.graph_state. A single streaming call can't be cut off mid-generation, so once the budget is reached the next turn is refused outright rather than the current one being truncated partway through.
Cost display covers only the live-streamed turn — the done SSE event's cost_usd is shown in the chat UI for the response as it streams in, but historical messages loaded after a page reload don't show it. agent_runs.message_id is set to the triggering user message, not the produced assistant message, so there's no direct join from a saved assistant message back to its cost — fixing that plus a message-list API field would be needed for full historical display.
Per-model pricing is a hardcoded, manually-maintained estimate — app/core/pricing.py is not sourced from a live pricing API. Useful for relative cost comparison and logging, not for billing reconciliation.
No re-ranking — Initial k=25 cosine retrieval is used directly. Adding a cross-encoder re-ranker would improve answer quality for ambiguous queries.
Web-search fallback is global, not per sub-query — Query decomposition can produce up to 5 sub-queries. The web-search trigger (< 3 relevant chunks) checks the total retrieved set, not per sub-query. A sub-topic with no document coverage can be missed if other sub-topics supply enough chunks to satisfy the global threshold.
Upload buffers entirely in memory — await file.read() loads the full file (up to 10 MB) before streaming to storage. There is no global concurrency cap on uploads, so simultaneous large uploads from different IPs can exhaust container memory. The fix is streaming upload directly to S3/R2 without buffering.
API key revocation is creator-only — DELETE /auth/api-keys/{id} filters on user_id, so only the key creator can revoke their own key. A workspace owner cannot revoke a key created by another member, even if that key is scoped to their workspace. Sufficient for single-user or trusted-team scenarios; a team product would require owner-level revocation as an additional query path.
All workspace roles can create API keys — Members with Viewer role can issue kb_ tokens for their workspace. This is intentional: all MCP endpoints are read-only (search, list_documents, get_document), so a Viewer-scoped key cannot write anything. If write tools are ever added to the MCP server, key issuance should be gated on Editor role or above.

Data Protection

Knowbase provides three endpoints for user data control:

Account deletion — `DELETE /auth/me`

Requires password confirmation in the request body (protection against session hijacking).

Three-case workspace rule:

Situation	Action
User is sole member of a workspace	Workspace deleted entirely (DB cascade + S3 files)
User is non-owner member of a shared workspace	Membership removed; workspace and other members' data are unaffected
User is owner of a shared workspace	409 Conflict — transfer ownership or delete the workspace first

To resolve the 409, invite another member as Owner first (Members page — owners can invite at any role, including Owner), then delete the account; deleting your own owner membership at that point auto-reassigns Workspace.owner_id to the remaining owner (see below). There is no separate "transfer ownership to a specific existing member" action — automatic canonical-owner reassignment happens as a side effect of removing an owner membership while another owner exists, not as its own dedicated workflow. S3 files from deleted workspaces are removed in a best-effort loop after the DB commit (failures are logged, not retried). Redis event-streams (sse:run:{id}) are not actively purged — they carry a 1 h TTL and expire automatically.

Data export — `GET /auth/me/export`

Returns a single JSON document containing all user-generated data:

Profile (id, email, role, created_at)
Workspace memberships (name + role)
Conversations and all messages
Memories (all scopes)
Document metadata (name, status, mime_type, size — not file contents)
API key metadata (name, created_at, last_used_at — never key hashes or raw keys)

Memory management — `GET /workspaces/{id}/memories`, `DELETE /memories/{id}`

GET /workspaces/{id}/memories lists all agent-extracted memories for a workspace (requires membership; visible to any role, not just the creator). DELETE /memories/{id} removes a specific memory — for user/global-scoped memories, only the creator may delete it; for workspace-scoped memories, the deleting user must currently hold editor role or above in that workspace, regardless of who originally created it (a member downgraded to viewer, or a non-creator editor, are handled correctly — see Architecture Decisions).

What the memory system collects: after each AI response, a lightweight extraction pass identifies facts, preferences, and context stated by the user and saves them as workspace-scoped memories. Memories are stored in the memories table, visible via the Memories module in the sidebar, and deletable at any time.

Roadmap

License

All rights reserved. This repository is published for portfolio and code-review purposes — to demonstrate the author's work to potential employers/collaborators — not as an open-source project. No license is granted to use, copy, modify, or redistribute this code.

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
.github/workflows		.github/workflows
backend		backend
frontend		frontend
infra		infra
mcp-server		mcp-server
.env.example		.env.example
.env.production.example		.env.production.example
.gitignore		.gitignore
.gitleaks.toml		.gitleaks.toml
.pre-commit-config.yaml		.pre-commit-config.yaml
README.md		README.md
docker-compose.prod.yml		docker-compose.prod.yml
docker-compose.yml		docker-compose.yml
fly.toml		fly.toml
railway.toml		railway.toml

Folders and files

Latest commit

History

Repository files navigation