tech-digest bot

A self-hosted bot that builds you a personalized morning tech digest. It continuously ingests primary sources (Hacker News, RSS, Reddit, GitHub releases, SEC EDGAR, Product Hunt, arXiv), clusters related items into stories, scores them against your interest profile, and every morning sends a single composed brief to your Telegram. Design principle: fast primary sources detect, slower press confirms — the bot fuses both.

• Personalized — you set the topics and weights; the digest reflects them.
• It learns — 👍/👎 buttons under each story tune what you see next.
• No repeats — a story shown once never appears again.
• Cheap & light — one Claude API call/day (~$0.05–0.10); clustering runs locally with a lightweight embedding model (no PyTorch), so it fits free hosts.
• Yours alone — everyone runs their own copy with their own keys. No shared server, nothing phones home. Apache-2.0 licensed.

What you get

Each morning: a short intro (the three things worth knowing if you read nothing else), then one message per story with 👍/👎 buttons right underneath. Tapping tunes tomorrow's ranking.

  

A real morning's digest. Each 🔴 / 🟠 / 🟢 / ⚪ tag is the story's "breaking level"; the italic "Why it matters to you" line ties each story back to your profile.

---

Quickest start — free, no server (recommended)

Want it delivered every morning without keeping a computer on, for free? Fork this repo, then follow HOSTING.md → GitHub Actions. You paste four keys into your repo's Actions Secrets and it runs itself daily in GitHub's cloud. No laptop, no server, ~15 minutes. (HOSTING.md also covers Replit, PythonAnywhere, and an always-on free Oracle VM for instant feedback.)

To try it locally first, or to develop it, follow the first-time setup below.

First-time setup (local)

1. Clone the repo and set up the virtualenv (needs Python 3.11+):

   git clone <repo-url> tech-digest && cd tech-digest
   python3 -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt

   The first run downloads a ~90 MB embedding model (fastembed / ONNX) used for clustering — once, then cached.

2. Get an Anthropic API key at console.anthropic.com. Fund it with a small prepaid credit and turn auto-reload OFF — the worst case is then "runs out of credit and stops," never a surprise bill.

3. Create a Telegram bot via @BotFather and note the bot token. Send your new bot any message first (Telegram only returns a chat's updates after you've messaged it), then run make chat-id to fetch your TELEGRAM_CHAT_ID.

4. Configure:

   cp .env.example .env

   Fill in ANTHROPIC_API_KEY, TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID, and EDGAR_CONTACT_EMAIL. Set TIMEZONE to yours. Leave the rest at defaults.

5. Initialize and smoke-test:

   make init-db
   make run-once      # ingest → cluster → score → compose → deliver, once

   Expect a digest in Telegram within a few minutes. The first day is thin by design — scoring needs a couple days of history to warm up.

6. Run it for real — pick a host from HOSTING.md. For the full-featured always-on path (instant 👍/👎, live /pause /status), run the persistent scheduler on a VPS / Raspberry Pi / free Oracle VM:

   make install-service && sudo systemctl enable --now tech-digest

   (Or make print-crontab for a cron-only schedule with no live commands.)

Personalize it

The digest is only as good as the profile behind it. Three files control it:

• config/profile.yaml — keyword weights for scoring (positive boosts a topic, negative suppresses it). The shipped list is an example — retune it.
• src/compose.py (SYSTEM_PROMPT) — the composer's tone and focus areas.
• config/sources.yaml — which feeds, subreddits, repos, and EDGAR companies get polled.

Your 👍/👎 taps refine the weights over time on top of whatever you set here, stored in a separate learned layer (keyword_adjustments) so your hand-set profile.yaml is never overwritten.

Known limitations

• A few seed RSS URLs in config/sources.yaml may be stale (feeds move) — a dead feed is logged and skipped, never breaking a cycle. Prune/update to taste.
• Reddit blocks unauthenticated JSON from many hosts (HTTP 403), so the Reddit source is often skipped. Reddit OAuth is a possible future add.
• Intraday breaking-news alerts are scaffolded but disabled (ALERTS_ENABLED=false); the breaking-level buckets are untuned defaults until there's a week of real data to calibrate against.
• On GitHub Actions (batch) hosting, 👍/👎 taps apply to the next morning's digest, not instantly — instant feedback needs an always-on host (see HOSTING.md).

Turning it off

Softest to nuclear. A stopped or paused bot costs exactly $0 in API spend.

Action	Effect	Cost
/pause from Telegram	Every stage no-ops before any network/API call; /resume restores	$0
make pause	Same pause-file, from the host	$0
systemctl stop tech-digest	Scheduler stops entirely	$0
make uninstall-service	Removes the systemd unit	$0
Revoke the API key	Even a misbehaving process can't spend	$0

Restarting after any pause is safe — every stage is idempotent, so it just catches up on the next cycle. (On GitHub Actions, "pause" is simply disabling the workflow in the Actions tab.)

Cost

Prepaid, auto-reload off, so a paused/stopped bot spends $0. Running normally: ~$0.05–0.10/day on claude-sonnet-5 (one composition call, plus an occasional retry). MAX_DAILY_SPEND_USD (default 0.50) is a bug tripwire, not a budget — crossing it skips further API calls and sends a Telegram warning. It should never trigger in normal use.

Makefile reference

Target	What it does
make init-db	Create/upgrade the SQLite schema
make run-once	Run the full pipeline once (ingest → cluster → score → compose → deliver)
make pause / make resume	Write / remove data/PAUSED
make chat-id	Look up your Telegram chat ID from recent bot messages
make print-crontab	Print a cron schedule (no live Telegram commands)
make install-service / make uninstall-service	Install / remove the systemd unit
make test	Run the test suite

Also runnable standalone for batch hosts: python -m src.run_once (one full pipeline pass) and python -m src.scheduler (the persistent always-on process).