chatstore

Local-first tool for ingesting personal message data (WhatsApp for macOS, Apple Messages), searching it via SQLite FTS5, citing entities through stable URNs, and exporting password-encrypted monthly archives that can rebuild the cache on another machine without either source app.

Durable contracts live in specs/; the system is described in docs/ (overview, architecture, sources, operations, security, integration, decisions, testing, history). The end-to-end acceptance test (scripts/acceptance.py) runs against real local sources.

Quick start (macOS)

Install from GitHub (needs Python ≥ 3.11):

pipx install git+https://github.com/lukeramsden/chatstore      # or: pip install git+https://github.com/lukeramsden/chatstore
chatstore init
chatstore helper install     # prebuilt Apple Messages decoder from GitHub releases, sha256-verified (no Rust needed)

Or work from a checkout:

git clone https://github.com/lukeramsden/chatstore && cd chatstore
python3 -m venv .venv && .venv/bin/pip install -e '.[dev]'
scripts/build-helper.sh              # Rust helper for Apple Messages bodies (needs cargo; rust-toolchain.toml pins the version)
.venv/bin/chatstore init             # data dir: ~/Library/Application Support/chatstore (or --data-dir / CHATSTORE_DATA_DIR)
.venv/bin/chatstore doctor           # Full Disk Access, schema compatibility, helper, FTS5
.venv/bin/chatstore sync             # first sync: minutes; later syncs: seconds
.venv/bin/chatstore search "see you tomorrow" --since 2025-01-01
.venv/bin/chatstore --json context urn:uuid:... --before 3 --after 3

Archives:

chatstore archive password suggest          # 256-bit password; store it in a password manager
chatstore archive password set              # macOS keychain (or CHATSTORE_PASSWORD_FILE=<0600 file>)
chatstore archive export -o ~/Archives      # catalogue + one WinZip-AES-256 ZIP per UTC month, text-only by default
chatstore archive verify ~/Archives
chatstore --data-dir /tmp/fresh archive import ~/Archives    # rebuilds a searchable cache with no source apps

Every command takes --json and returns the chatstore-cli-v1 envelope (specs/cli-json.md). Exit codes: 0 ok · 1 failure · 2 usage · 3 permission · 4 incompatible · 5 partial · 6 conflict · 7 invalid archive · 8 not found · 9 not initialised.

Agent skill

An agent skill that uses only the CLI lives in skills/chatstore/SKILL.md. Install it with the skills CLI for any supported agent (Claude Code, Cursor, Codex, Pi, …):

npx skills add lukeramsden/chatstore              # interactive: pick agents and scope
npx skills add lukeramsden/chatstore -g -y        # global install, no prompts

No separate setup is needed: the skill ships with scripts/install.sh, which the agent runs on first use if chatstore is missing. It installs the CLI from GitHub, runs chatstore init, downloads the verified Apple Messages helper and runs chatstore doctor. You will be asked to grant Full Disk Access and to approve the first chatstore sync. The skill never reads the cache or source databases directly; exports, imports, identity links and purges require your approval.

Layout

src/chatstore/            Python package
  adapters/               read-only source adapters (whatsapp, messages), snapshotting
  canonical/              canonical record model, JSON schemas, schema version
  identity/               UUIDv5 URNs, account scopes, revision digests
  cache/                  SQLite schema, migrations, FTS5, store, queries
  archive/                container, manifest, export / verify / import, password
  cli/                    command surface + JSON envelope
  sync.py                 sync orchestrator; curation.py: links, scopes, conflicts, purge
helpers/messages-decoder/ Rust helper (pinned imessage_database) -> versioned JSONL
specs/                    contracts, JSON Schemas, test vectors, source findings
docs/                     final-state documentation (see docs/README.md)
scripts/                  build-helper.sh, acceptance.py (final acceptance test)
skills/chatstore/         portable agent skill (CLI client only)
tests/                    unit / integration / synthetic fixtures (no real data)

Development

.venv/bin/pytest -q            # synthetic fixtures only; no real data required
.venv/bin/ruff check src tests scripts
.venv/bin/mypy src
.venv/bin/python scripts/acceptance.py --media-month 2026-04   # real sources, temp data dirs, ~25 min

Rules

• Never modify source databases. Open read-only; snapshot via the SQLite backup API.
• Never commit passwords, raw messages, cache files, archives, or real-data fixtures.
• Never change identity derivation rules silently; version them (chatstore-id-v1).
• Passwords never on argv, in env values, logs or JSON.