Skip to content

feedsnap v0.4.0 — SQLite deduplication

Choose a tag to compare

View all tags
@rook-builds rook-builds released this 13 Jul 06:32
v0.4.0
66077aa

What's new

feedsnap now remembers what it's already shown you.

Pass --dedup on any run and feedsnap tracks seen entry URLs in ~/.feedsnap/seen.db. On the next run, only genuinely new entries come through. No database to set up — it creates itself on first use.

# First run: shows everything, marks it all seen
feedsnap https://lobste.rs/rss --dedup

# Second run: only new entries since last time
feedsnap https://lobste.rs/rss --dedup

# Custom DB — per-project seen lists
feedsnap https://lobste.rs/rss --seen-db ~/myproject/feeds.db

Works in OPML multi-feed mode too. Each feed maintains its own seen set, scoped by URL.

Details

  • --dedup: use ~/.feedsnap/seen.db (created automatically)
  • --seen-db PATH: custom path (implies --dedup)
  • Schema: seen_entries(url TEXT PRIMARY KEY, feed_url TEXT, first_seen TEXT)
  • INSERT OR IGNORE — repeated runs are idempotent, no duplicates
  • No new runtime dependencies (sqlite3 is stdlib)
  • New public module: feedsnap.seen_db with get_seen(), mark_seen(), default_db_path()

Upgrade

pip install --upgrade feedsnap

Full changelog: https://github.com/rook-builds/feedsnap/blob/main/CHANGELOG.md

Assets 2
Loading