wiki

A local, markdown-first personal wiki with AI-assisted ingestion, semantic indexing, retrieval, and chat.

What it does

• Turns raw source markdown in raw/ into structured notes in wiki/.
• Builds a reusable vector index at .wiki-index/index.json.
• Answers grounded questions over your notes with citations.
• Runs an interactive chat over the same retrieval pipeline.

Quick start

1. Install dependencies:

pnpm install

2. Create environment file:

cp .env.example .env

3. Set your API key in .env:

OPENAI_API_KEY="..."

4. Ingest a raw markdown file:

pnpm ingest raw/ai/large-language-model.md

5. Build/update the index:

pnpm index

6. Query:

pnpm query "What are the main limitations of LLMs?"

7. Start chat:

pnpm chat

Command reference

pnpm ingest <raw-file> [options]

Converts a markdown file under raw/ into a structured note under wiki/, preserving the relative path.

Options:

• --model <id>: generation model (default: WIKI_MODEL, fallback gpt-4.1-mini).
• --max-chars <n>: max source chars to ingest (default: 100000).

Example:

pnpm ingest raw/philosophy/nihilism.md --model gpt-5-mini --max-chars 80000

pnpm index [options]

Builds or updates the semantic index from files in wiki/.

Options:

• --embedding-model <id>: embedding model (default: WIKI_EMBEDDING_MODEL, fallback text-embedding-3-small).
• --chunk-size <n>: chunk size in characters (default: 1100, must be > 200).
• --chunk-overlap <n>: overlap in characters (default: 180, must be >= 0).
• --batch-size <n>: embedding batch size (default: 32).
• --index-path <path>: custom index file path.

Example:

pnpm index --chunk-size 1200 --chunk-overlap 200

pnpm query <question> [options]

Retrieves top chunks and produces a grounded answer with confidence and source references.

Options:

• --model <id>: answer model (default: WIKI_ANSWER_MODEL, fallback gpt-5-mini).
• --embedding-model <id>: query embedding model (default: WIKI_EMBEDDING_MODEL).
• --k <n>: number of retrieved chunks (default: 8).
• --index-path <path>: custom index file path.

Example:

pnpm query "Compare absurdism and nihilism" --k 10

pnpm chat [options]

Starts interactive retrieval chat using the same models/index options as query.

Chat commands:

• /sources: show citations from the last answer.
• /reset: clear conversation history.
• /exit: quit chat.

Environment variables

See .env.example:

OPENAI_API_KEY=""
WIKI_MODEL="gpt-5-mini"
WIKI_ANSWER_MODEL="gpt-5-mini"
WIKI_EMBEDDING_MODEL="text-embedding-3-small"

Notes:

• OPENAI_API_KEY is required for ingestion, indexing (when new embeddings are needed), query, and chat.
• CLI flags override env vars.

Project structure

raw/          # source markdown clips
wiki/         # generated canonical notes
lib/          # ingestion, indexing, retrieval, chat modules
scripts/      # CLI entrypoint
.wiki-index/  # generated embedding index

Typical workflow

1. Add or update markdown files in raw/.
2. Run pnpm ingest <raw-file> for each new/changed source.
3. Run pnpm index to refresh embeddings incrementally.
4. Use pnpm query or pnpm chat to explore your wiki.