Tom's Knowledge Base

A personal knowledge management system combining notes, blog, and a reference library with 200k+ bookmarks.

Overview

This project is a Markdown-first, Git-backed knowledge base with:

• Notes - Zettelkasten-style interconnected notes
• Blog - Chronological posts with RSS feeds
• References - Citation database in sharded JSONL format
• Code Snippets - Canonical examples with registry
• AI/RAG Ready - Machine-readable indices for LLM consumption
• Pluggable SSGs - Multiple views (MkDocs Material + Quarto)

Architecture

See SPEC.md for the complete specification and AGENTS.md for development workflow.

Key Components

• Content: notes/ (Markdown), data/refs/ (JSONL references)
• Tools: tools/*.py (validation, generation, export)
• SSG Adapters: ssg/{mkdocs,quarto}/adapter.sh
• AI Guidance: ai/llm.txt, ai/site-index.json
• Build System: justfile (modern task runner)

Quick Start

Prerequisites

• Python 3.12+
• uv for package management
• just for task running (optional but recommended)
• MkDocs Material for SSG
• Quarto for alternative SSG view

Installation

# Install Python dependencies
uv sync

# Install MkDocs and plugins
pip install mkdocs-material mkdocs-material-extensions \
  mkdocs-blog-plugin mkdocs-bibtex \
  mkdocs-roamlinks-plugin mkdocs-awesome-pages-plugin

# Install Quarto (see https://quarto.org/docs/get-started/)

Build the Site

# Validate all content
python3 tools/validate_refs.py
python3 tools/check_front_matter.py
python3 tools/extract_snippets.py --check-only

# Generate derived artifacts
python3 tools/build_refs.py
python3 tools/mkdocs_pages.py
python3 tools/export_ai_index.py

# Build MkDocs view
./ssg/mkdocs/adapter.sh build

# Or use justfile (if installed)
just validate
just generate
just build

Preview Locally

# MkDocs
./ssg/mkdocs/adapter.sh serve

# Quarto
./ssg/quarto/adapter.sh serve

# Or both (requires just)
just serve

Development

Quick Start with Dev Container (Recommended)

1. Install Docker and VS Code
2. Install the Dev Containers extension
3. Open this project in VS Code
4. Click "Reopen in Container" when prompted
5. Wait for setup to complete (~2-3 minutes first time)

Local Development Setup

Set up everything with a single command:

# One-command setup: installs everything you need
./scripts/setup-environment.sh

# Verify everything is ready
./scripts/check-environment.sh

This automatically installs:

• uv package manager
• Python 3.12
• External tools (shellcheck, shfmt, etc.)
• Python dependencies
• Tox for test orchestration
• Pre-commit hooks

Testing & Quality Checks

# Run all checks (linting, formatting, tests)
uvx tox

# List all available environments
uvx tox list

# Specific checks
uvx tox -e ruff          # Python linting
uvx tox -e pytest        # Run tests
uvx tox -e quality       # Quality tests

Pre-commit Hooks

# Install hooks
uv run pre-commit install

# Run manually
uv run pre-commit run --all-files

Requirements

• Python 3.12+
• uv for package management

Project Structure

notes/
├── docs/                      # USER-FACING CONTENT
│   ├── index.md              # Site homepage
│   ├── notes/                # Knowledge base notes
│   │   ├── blog/            # Blog posts
│   │   └── *.md             # Individual notes
│   └── references/           # Generated reference pages (gitignored)
├── dev/                       # PROJECT DOCUMENTATION
│   ├── IMPLEMENTATION_SUMMARY.md
│   └── session-commands.md   # Command history
├── data/
│   ├── refs/                 # Reference database (JSONL)
│   └── derived/              # Generated CSL/BibTeX (gitignored)
├── code/                      # Canonical code examples
├── ai/                        # AI/LLM guidance files
├── ssg/                       # SSG adapters (mkdocs, quarto)
├── tools/                     # Python build tools
├── src/notes/                 # Python package for tooling
├── tests/                     # Test suite
├── .github/workflows/         # CI/CD pipelines
├── justfile                   # Task runner recipes
├── mkdocs.yml                 # MkDocs configuration
├── _quarto.yml                # Quarto configuration
├── SPEC.md                    # Project specification (root for discoverability)
├── AGENTS.md                  # Agent development guide (root for discoverability)
└── README.md                  # This file

Content Organization

Notes (docs/notes/)

Personal knowledge base entries with front matter:

---
id: unique-id
title: Note Title
date: 2025-01-01
tags: [tag1, tag2]
---

# Content

Citations: [@reference-id]
Wikilinks: [[Other Note]]

Blog (docs/notes/blog/)

Chronological posts with date-prefixed filenames:

• 2025-01-01-post-title.md
• RSS/Atom feeds generated automatically

References (data/refs/shards/)

JSONL reference database, sharded by hash:

{"id":"smith2025-example","type":"web","title":"...","url":"https://...","authors":["Smith, J."],"year":2025,"tags":["tag"]}

Code (code/)

Canonical examples with snippet registry (SNIPPETS.yaml) for inclusion in notes.

Project Documentation (dev/)

Developer and project documentation:

• IMPLEMENTATION_SUMMARY.md - What was built and how
• session-commands.md - Command history from setup

Note: SPEC.md and AGENTS.md remain at root for discoverability by AI agents and contributors.

Specification-Driven Development

This project follows spec-driven development using GitHub Spec Kit:

1. Features defined in SPEC.md
2. Implementation guided by AGENTS.md
3. Tasks tracked via GitHub issues labeled spec:ready
4. PRs reference issues and include acceptance tests

Agent Workflow

Multiple AI agents (Amazon Q, Claude Code, Codex) can work in parallel:

• Each agent claims issues in their domain (see AGENTS.md)
• Agents follow conventional commits and PR checklists
• Integration tested via just build before merge

See AGENTS.md section 11 for details.

CI/CD Pipeline

GitHub Actions workflow:

1. Validate - Schema, front matter, snippets
2. Generate - CSL JSON, BibTeX, reference pages, AI index
3. Build - Both MkDocs and Quarto views
4. QA - Link check (lychee), prose (Vale), accessibility (pa11y)
5. Deploy - GitHub Pages

Roadmap

• v0.1 - Skeleton, validators, MkDocs, basic reference index ← You are here
• v0.2 - Quarto view, AI artifacts, snippet integration
• v0.3 - Importers (Chrome/Firefox/Zotero), nightlies, feeds
• v1.0 - Scale test with ≥50k refs, split to submodule

License

• Content (notes, blog): CC BY-SA 4.0
• Code: MIT License

Acknowledgments

• Project structure from Python Project Template
• Spec-driven approach inspired by GitHub Spec Kit