Skip to content

IceWizard98/wiki-ai

Repository files navigation

WikiAI

License: MIT Python GitHub stars GitHub issues

An AI-generated Italian encyclopedia. Users search for any topic: if the article doesn't exist, it is generated on-demand by an LLM, enriched with internal wikilinks, tags, and backlinks to existing articles. Generation runs in the background; the browser polls for completion and redirects when the article is ready.

Features

  • On-demand generation — every unknown topic triggers an async LLM pipeline
  • Bidirectional wikilinks — forward linker (LLM + slug normalization) + backward linker (regex) connect articles to each other
  • Search with autocomplete — dropdown over existing articles, inline creation for new topics
  • Navigation — tag pages (/tag/<tag>), A–Z index (/indice), random article
  • Dark mode — persistent toggle + prefers-color-scheme auto-detection
  • Mobile-first — TOC drawer, reading progress bar, back-to-top FAB
  • SEO / GEOrobots.txt, llms.txt, Schema.org JSON-LD (Article + BreadcrumbList + WebSite), canonical URLs, og:image, <time datetime>, XML sitemap with static pages

Stack

Layer Technology
Web FastAPI + Jinja2
CSS Tailwind CSS v3 (standalone CLI build)
Database SQLite + sqlite-vec (embedding similarity)
Embeddings sentence-transformers (paraphrase-multilingual-mpnet-base-v2)
LLM Anthropic Claude / OpenAI / Ollama (configurable)
Container Docker multi-stage (css-builder → py-builder → runtime)

Quick start

Docker Compose (full stack with Ollama)

cp .env.example .env   # or create .env manually
docker compose up -d

App available at http://localhost:8000. Ollama pulls the model on first start (see docker/ollama-entrypoint.sh).

Cloud LLM (Anthropic or OpenAI)

cp .env.example .env
# Set LLM_PROVIDER and the API key in .env
docker compose up wiki-ai -d   # app only, no Ollama

Local development

pip install -e ".[dev]"
# Build CSS (requires Tailwind standalone CLI in PATH)
tailwindcss -i static/css/input.css -o static/css/main.css --watch

uvicorn app.main:app --reload

Environment variables

Variable Default Description
LLM_PROVIDER anthropic anthropic | openai | ollama
LLM_MODEL claude-sonnet-4-6 Main model (writer)
LLM_MODEL_FAST (same as LLM_MODEL) Fast model for quick steps (metadata, linker)
ANTHROPIC_API_KEY Required when LLM_PROVIDER=anthropic
OPENAI_API_KEY Required when LLM_PROVIDER=openai
OLLAMA_BASE_URL http://localhost:11434 Ollama endpoint
EMBEDDING_PROVIDER local local (sentence-transformers) | openai
SITE_BASE_URL http://localhost:8000 Public site URL — used in sitemap, canonical, og:image
SECTION_PARALLELISM 4 Number of sections written in parallel

Architecture

Hexagonal architecture (Ports & Adapters):

app/
├── domain/          # Article, Outline — pure domain objects, no dependencies
├── ports/           # Interfaces: LLMClient, EmbeddingClient, ArticleRepository, …
├── adapters/        # Implementations: AnthropicClient, OllamaClient, SQLiteArticleRepo, …
├── services/        # ArticleService, BacklinkService, RenderService
├── pipeline/        # Orchestrator + LLM pipeline steps
│   └── steps/       # TitleSlug → Outline → SectionWriter → Metadata → ForwardLinker
└── web/             # FastAPI router, Jinja2 templates, static JS

Generation pipeline

topic
  ↓ TitleSlugStep       — canonical title + slug
  ↓ OutlineStep         — section structure (JSON)
  ↓ SectionWriterStep   — Markdown body (parallel)
  ↓ MetadataStep        — description + tags
  ↓ ForwardLinkerStep   — inserts [[slug|anchor]] links to existing articles
  → persists to disk (.md) + SQLite + sitemap
  ↓ BacklinkService     — patches existing articles with links back to the new one

Wikilinks ([[slug|text]]) are resolved to HTML at render time by RenderService.

Tests

pytest                                  # all tests
pytest tests/test_render_service.py    # single file
pytest -k "test_backward"              # filter by name

Production notes

  • Docker volumes: ./pages (Markdown source) and ./data (SQLite + sitemap) must be persistent.
  • First start with Ollama takes 2–5 minutes to pull the model.
  • The embedding model (~400 MB) is pre-downloaded at Docker build time.
  • Set SITE_BASE_URL to the real public URL for correct canonical URLs and sitemap entries.

About

No description, website, or topics provided.

Resources

License

Stars

3 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors