Skip to content

sembr 1.0.0 — initial public release

Choose a tag to compare

View all tags
@KaihuaHuang KaihuaHuang released this 16 May 08:00
· 108 commits to main since this release
v1.0.0
197103c

sembr 1.0.0 — Reverse RAG, self-hosted.

sembr is a self-hosted intent radar. You describe what you care about once — "monitor Fed policy impact on emerging-market currencies" — and it continuously scans RSS feeds, news APIs, and social streams, matches articles to your intent via semantic vectors, and delivers LLM-analyzed digests from whatever angle you configure.

Apache-2.0. ~1 GB total image footprint. Runs on a 4 GB box.

git clone https://github.com/Peakstone-Labs/sembr.git
cd sembr && cp .env.example .env   # set EMBEDDER_API_KEY + SMTP creds
docker compose up --build

Or — paste this to your AI coding agent:

Read https://github.com/Peakstone-Labs/sembr/blob/main/agent/INSTALL.md and follow it to install sembr on this machine.

15 minutes from git clone to your first email digest. Live demo: panel.peakstone-labs.com/#news — what sembr is producing today.

What's in the box

Dashboard

  • Reverse-RAG matcher — cron + event modes; per-intent threshold (0.60–0.95); two-layer dedup (MD5 ingest + per-intent match_seen).
  • 53 pre-loaded sources across three source types — 22 RSS feeds (Guardian / SCMP / NPR / Bloomberg Markets / 华尔街见闻 / 第一财经 / 36氪 / Nature ×3 / HelloGitHub / …), 30 NewsAPI.ai aggregator sources (Reuters / BBC / NYT / WSJ / FT / Economist / TechCrunch / Wired / …), 1 Twitter feed via bundled RSSHub sidecar.
  • BGE-M3 embeddings on SiliconFlow — 1024-dim, 8192-token context, native bilingual EN/CN. Free at any volume on the BGE-M3 tier.
  • DeepSeek-V4-Flash summaries — 1M-token context lets one digest ingest a hundred long articles for well under a cent.
  • Custom prompt templates — system + instruction templates with strict-placeholder validation; per-intent template selection; edits take effect on the next scheduler tick.
  • Email digest channel — SMTP multipart/related, inline assets, matcher-score badges, rendered in each intent's own timezone. Telegram / Discord / Slack scaffolded for post-1.0.
  • Qdrant 1.17.1 dual-collection design (intents + news) with scalar int8 quantization, always_ram=True ANN over disk-resident raw vectors.

Intents

Built for agents

sembr was designed agent-first from day one:

  • agent/INSTALL.md — a 6-phase install protocol written for AI coding agents to read. Paste the URL to Claude Code / Cursor / Cline / Aider / Continue / Roo and walk away for 15 minutes.
  • agent/sembr/ — a full Agent Skills bundle (SKILL.md + endpoint / schema / recipe / error references). Drop it into ~/.claude/skills/sembr/ and your agent can drive a running sembr instance: create intents, tune thresholds, edit templates, trigger diagnostic fires.
  • POST /api/external/intents/{id}/fire — synchronous endpoint returning matched articles + LLM summary in one round-trip. No notification side-effects, no match_seen writes, per-call overrides for lookback / threshold / feed scope. Hermes / OpenClaw / LangGraph / your own orchestrator can treat sembr as a tool node.

Deploy sembr with an agent. Drive it as an agent's tool. Both work out of the box.

Why "Reverse RAG"?

In classic RAG, you ask a question; the system retrieves and answers.

In Reverse RAG, you describe a long-running intent once; the system retrieves forever, scanning incoming articles against your intent vector on a schedule you set.

Attention Is All You Need — AI is your attention.

The matcher is built literally on attention-based BGE-M3 embeddings. Out the back: the digest comes through whatever framing you wrote into the prompt template. Same engine, different angle per intent.

Live demo

panel.peakstone-labs.com/#news — two production intents (霍尔木兹海峡危机日报 / AI 算力日报) running on sembr right now, regenerated daily, anyone can read. What's on screen is what sembr produces.

Right-sized

Component Memory (default 53 sources) mem_limit (4× headroom)
api ~125 MiB 1.5G
qdrant ~520 MiB 2G
rsshub ~355 MiB 512M
Total ~1 GB —

Homelab / Mac mini / NAS / $10 VPS all comfortable. First-pull total ~1 GB (Python 3.12 base + Qdrant + RSSHub).

Security model

  • DashboardTokenMiddleware gates /api/* behind X-Dashboard-Token. /health is intentionally auth-free.
  • sembr logs an ERROR if DASHBOARD_TOKEN is empty — OK for localhost dev, unsafe for any host reachable beyond loopback.
  • See docs/deployment/public.md for the end-to-end public-deployment guide: reverse-proxy samples (Caddy / nginx / Cloudflare Tunnel), ufw, SSH hardening, nmap verification.
  • Private Vulnerability Reporting enabled — please report security issues via the Security tab, not public issues. See SECURITY.md.

Tech stack

Python 3.12 · FastAPI 0.115 · Pydantic v2 · APScheduler 3.11.2 · Qdrant 1.17.1 · aiosqlite 0.20 · httpx 0.27 · feedparser 6 · BGE-M3 (SiliconFlow) · DeepSeek-V4-Flash (SiliconFlow).

Links

Acknowledgments

Built by Peakstone Labs. sembr is our first open-source project — and not the last.

We're a quant research outfit. The problems sembr solves (information overload, unstructured-source tracking, agent-callable monitoring) are the ones we hit ourselves every day. Open-sourcing them is the cheapest way to make them better — for us and for everyone facing the same problems.

License: Apache-2.0 · See NOTICE for attribution.

Issues: GitHub Issues · Discussion: GitHub Discussions · Security: PVR

Assets 2
Loading