Trace

Trace is a local-first RAG knowledge base with evaluation. It ingests PDFs, Markdown, and text files, chunks them, embeds them, stores them in Qdrant, answers questions with citations, and measures whether retrieval and generation are improving.

What it shows

• Document ingestion for PDF, Markdown, and text.
• Configurable chunking strategies: fixed token windows, Markdown heading-aware chunks, and sentence windows with overlap.
• OpenAI-compatible provider adapter with Ollama defaults and optional cloud providers.
• Citation-first question answering with retrieved evidence and latency breakdowns.
• Evaluation dashboard for retrieval precision, hallucination rate, faithfulness, latency, and chunking strategy comparison.

Architecture

flowchart LR
  U["User"] --> W["Next.js workbench"]
  W --> A["Go REST API"]
  A --> S[("SQLite metadata")]
  A --> F["Local uploads"]
  A --> O["OpenAI-compatible model provider"]
  A --> Q["Qdrant vector database"]
  A --> E["Evaluation engine"]
  E --> S
  E --> O
  E --> Q

Loading

Local development

1. Install Go, Node.js, pnpm, Docker, and Ollama.
2. Copy .env.example to .env.
3. Start Qdrant:

docker compose up -d qdrant

4. Pull local models:

ollama pull llama3.1:8b
ollama pull nomic-embed-text

5. Install dependencies and run the app:

pnpm install
pnpm dev

The API runs on http://localhost:8080, the web app runs on http://localhost:3000.

API surface

• POST /api/documents uploads a PDF, Markdown, or text file.
• GET /api/documents lists documents and ingestion state.
• GET /api/documents/{id} returns document detail and chunks.
• DELETE /api/documents/{id} removes metadata and local upload records.
• POST /api/questions answers with citations and latency metrics.
• POST /api/evals/datasets creates an evaluation dataset.
• POST /api/evals/runs runs evaluation cases.
• GET /api/evals/runs lists evaluation runs.
• GET /api/evals/runs/{id} returns run metrics and case results.

Evaluation Metrics

• Retrieval precision@k: how many expected documents or chunks appear in top-k retrieval.
• Hallucination rate: share of answers judged unsupported by retrieved context.
• Faithfulness: support score from evidence-grounded grading.
• Latency: retrieval, generation, and end-to-end timings.
• Chunking strategy comparison: aggregate quality and speed grouped by strategy.

Scope notes

Trace supports selectable-text PDFs. OCR for scanned documents is intentionally out of scope.

License

MIT