FutureFeed

Given what has been written about over the past N weeks, what topics are most likely to emerge next?

FutureFeed turns RSS feed history into a causal topic graph and uses it to predict the next 1–3 weeks of content — with confidence scores and structured signals ready to hand to an LLM.

→ Live visualization

---

How it works

1. Cluster — weekly posts are grouped into topic threads using sentence embeddings + agglomerative clustering
2. Graph — topic threads are connected across weeks by causal edges (weighted cosine similarity, multi-lag up to 6 weeks)
3. Predict — a centroid-similarity forward projection scores candidate topics for B+1, B+2, B+3 horizons
4. Visualize — an interactive D3.js timeline shows the full causal graph + predictions with live filter controls

The prototype runs on ~12 months of AWS blog RSS feeds (~20 topic-specific feeds).

Quickstart

pip install -r requirements.txt

# Run full pipeline + open visualization
python src/pipeline.py aws_blog_rss_lastyear.csv --predict

# Rebuild visualization only (if graph already exists)
python src/visualize.py aws_blog_rss_lastyear.csv \
  --graph output/aws_blog_rss_lastyear/causality_graph.json

Visualization controls

Control	What it does
Min edge weight slider	Hide weaker causal links
Max lag dropdown	Show only short-range or long-range arcs
Nodes/col slider	Show more or fewer topic clusters per week
B+1 / B+2 / B+3 checkboxes	Toggle prediction horizons

Project layout

src/
  pipeline.py      # end-to-end runner
  config.py        # all hyperparameters (single source of truth)
  clustering.py    # weekly topic clustering
  causality.py     # causal graph construction
  predict.py       # multi-horizon forward prediction
  visualize.py     # D3.js HTML output
  search.py        # parallel hyperparameter search
feeds/             # per-topic RSS CSVs
output/            # generated graphs, summaries, HTML
docs/              # GitHub Pages (latest visualization)
DESIGN.md          # detailed design document

Key hyperparameters

All tunable parameters live in src/config.py. Optimised defaults (90% hit-rate on held-out data):

Parameter	Default	Effect
distance_threshold	0.45	Cluster granularity
alpha / beta	0.85 / 0.15	Edge weight: similarity vs momentum
lag_decay	0.85	Penalty per additional lag week
min_edge_weight	0.10	Graph pruning threshold

See DESIGN.md for the full design rationale.