Skip to content

MLDreamer/ai-feed

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ai-feed 🧠

Your personal AI intelligence feed. Zero cost. Adversarially verified.

python run.py --topic "LLM agents"

One command. Verified digest. No subscriptions. No cloud. No cost.


What this is

A four-stage Python pipeline that fetches AI research from arXiv, RSS feeds, and HuggingFace Papers, ranks it by relevance to your topic, extracts one factual claim per item, and runs a reasoning model to verify whether each claim is actually supported by the evidence cited.

Every item gets a label: VERIFIED, DISPUTED, or UNVERIFIED.

The output is a structured markdown report saved to digest_output/.


What this is not

  • Not a RAG system. It does not index your documents.
  • Not a chatbot. You cannot ask follow-up questions.
  • Not real-time. arXiv has a 24-hour lag.
  • Not a memory system. Each run starts fresh.
  • Not a full-paper reader. It works from abstracts and summaries.

These are honest limitations, not bugs. The architecture is intentionally minimal.


The pipeline

arXiv API ──┐
RSS feeds ──┼──► fetch_all() ──► rank_items() ──► summarize_all() ──► verify_all() ──► digest
HF Papers ──┘
               (zero cost)    (priority score)  (fast model)     (reasoning model)

Stage 0 — Fetch. Pulls from three sources. No LLM calls. Zero cost.

Stage 0.5 — Rank. Scores each item: S = Authority + Semantic Density - Temporal Decay. ArXiv papers score highest (60), HuggingFace papers next (40), RSS last (20). Each keyword match in the title adds 25. Transparent weights you can change in two lines.

Stage 1 — Summarize. Fast model (Gemini Flash / Qwen2.5) extracts one factual claim and the verbatim evidence supporting it. Does not verify. Just extracts.

Stage 2 — Verify. Reasoning model (DeepSeek-R1) receives only the claim and evidence. It cannot see Stage 1's confidence. It applies five rules and returns a verdict.

Stage 3 — Digest. Formats everything as structured markdown. Saves to digest_output/.


The five verification rules

  1. Evidence directly supports claim with numbers or verbatim quotes → VERIFIED
  2. Claim uses a superlative not present in the evidence → DISPUTED
  3. Single social media source, no corroboration → UNVERIFIED
  4. Benchmark claim without benchmark name AND score → UNVERIFIED
  5. Blog post or press release without linked primary data → UNVERIFIED

Hard rules. Not soft heuristics. The verifier cannot decide to be lenient.


Quick start (5 minutes)

# 1. Get a free API key at openrouter.ai

# 2. Clone and install
git clone https://github.com/your-handle/ai-feed
cd ai-feed
pip install -r requirements.txt

# 3. Add your key
cp .env .env.local
# edit .env.local: OPENROUTER_API_KEY=your_key_here

# 4. Run
python run.py --topic "speculative decoding"

Fully local (no API key, needs ~12GB RAM)

curl https://ollama.ai/install.sh | sh
ollama pull qwen2.5:7b
ollama pull deepseek-r1:7b
# edit scripts/llm.py: switch to ollama backend
python run.py --topic "transformer architecture"

Topic profiles

python run.py --profile profiles/ml_engineer.yaml
python run.py --profile profiles/fmcg_data_scientist.yaml
python run.py --profile profiles/agent_builder.yaml

Edit any profile in profiles/. Add your topics, your RSS feeds, your email.


Sample output

══════════════════════════════════════════════════════════════
AI FEED — LLM AGENTS — 2026-05-10
══════════════════════════════════════════════════════════════

🟢 VERIFIED CLAIMS
──────────────────
📄 EAGLE3: 3.1x inference speedup with zero accuracy loss on LLaMA-3-70B
   Claim: wall-clock speedup 3.1x, perplexity unchanged at 5.68
   Source: arxiv.org/abs/2605.07891

🟡 DISPUTED CLAIMS
──────────────────
📄 "Our method outperforms all existing speculative decoding approaches"
   Flag: Superlative not supported. Paper compares to 3 methods, not comprehensive survey.

⚪ UNVERIFIED
─────────────
📄 OpenAI planning native speculative decoding in API this quarter
   Source: twitter.com — single source, no corroboration
══════════════════════════════════════════════════════════════
1 verified / 1 disputed / 1 unverified
══════════════════════════════════════════════════════════════

Automate

# Every morning at 7am
0 7 * * * cd /path/to/ai-feed && python run.py --profile profiles/ml_engineer.yaml

Hardware

Machine Recommended setup
8GB RAM laptop OpenRouter free tier (default)
16GB RAM / M1 Mac Ollama: qwen2.5:7b + deepseek-r1:7b
32GB RAM Ollama: qwen2.5:14b + deepseek-r1:14b

Files

ai-feed/
├── run.py                     ← entry point
├── scripts/
│   ├── llm.py                 ← LLM interface (OpenRouter / Ollama)
│   ├── fetch.py               ← fetch + rank
│   ├── summarize.py           ← Stage 1: claim extraction
│   ├── verify.py              ← Stage 2: adversarial verification
│   └── digest.py              ← Stage 3: markdown output
├── profiles/
│   ├── ml_engineer.yaml
│   ├── fmcg_data_scientist.yaml
│   └── agent_builder.yaml
├── digest_output/             ← auto-created on first run
├── .env
└── requirements.txt

To understand the what why and how,read this.

What comes next

  • Cross-session memory and contradiction detection
  • Full PDF verification (not just abstracts)
  • Source reputation scoring
  • Citation velocity weighting via Semantic Scholar API
  • Web interface

Pull requests welcome on all of these.


License

MIT. Fork it. Extend it. Use it commercially.


If this saves you time, star it. That is the only ask.

About

A 4-stage adversarial research auditor that fetches papers from arXiv & HuggingFace, extracts claims, and uses DeepSeek-R1 to verify them against raw abstracts.A self-correcting research & paper digest pipeline powered by local LLMs & reasoning agents

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages