readwise-reader-sync

Syncs documents from the Readwise Reader API v3 into a PostgreSQL database. Runs as a short-lived CLI — invoke it on a schedule (cron, systemd timer, etc.) to keep your database up to date.

Features

• Full or incremental sync via --full-sync flag
• Resumable: records a checkpoint after each successful run and only fetches documents updated since then
• Idempotent upserts — safe to run repeatedly
• Automatic retries on rate-limit (429) and server errors (5xx) with Retry-After header support
• Change history on the reading table via a PostgreSQL audit trigger

Requirements

• Rust (stable)
• PostgreSQL
• A Readwise access token

Tool versions are pinned in mise.toml. Install with mise:

mise install

Configuration

Create a .env file in the project root (or export the variables directly):

DATABASE_URL="postgres://postgres:password@localhost/readwise"
READWISE_ACCESS_TOKEN="<your token>"
RUST_LOG=info

If you use direnv, the included .envrc loads .env automatically.

Usage

# Incremental sync (default — uses saved checkpoint)
cargo run

# Full sync — ignore checkpoint and re-fetch everything
cargo run -- --full-sync

On first run with an empty database, a full sync is performed automatically regardless of the flag.

Database Schema

Migrations run automatically at startup. The schema consists of three tables:

Table	Purpose
reading	One row per Readwise document
sync_state	Single-row checkpoint storing the last successful sync timestamp
history	Audit log of all changes to the reading table

The reading table uses two PostgreSQL ENUM types: category (article, email, epub, highlight, note, pdf, rss, tweet, video) and location (archive, feed, later, new, shortlist).

Development

# Install dev tooling (one-time)
just dev-install

# Run with live database
cargo run

# Lint (warnings treated as errors, matching CI)
cargo clippy --all-targets --all-features -- -D warnings

# Format
cargo fmt

# After modifying any sqlx::query! macro: regenerate the offline cache
cargo sqlx prepare

The .sqlx/ directory contains a pre-built query cache committed to the repository so that Docker builds work without a live database (SQLX_OFFLINE=true). Always run cargo sqlx prepare and commit the updated cache after changing any SQL query.

Pre-commit hooks (via pre-commit, run on push) enforce cargo fmt, cargo clippy, conventional commits, and TOML/Markdown formatting.

Deployment

The project builds to a minimal Debian-based Docker image using a multi-stage build with cargo-chef for layer caching. Images are pushed to Scaleway Container Registry.

just build   # prepare sqlx cache + build + tag image
just push    # push to registry
just all     # build + push

Images are tagged with the current git tag, or the short commit hash if no tag exists.