SubTracker

The open-source Substack analytics & competitor intelligence platform

Paste any Substack URL. Get real data, AI-powered insights, and competitor analysis in seconds.

---

What is SubTracker?

SubTracker is a fully functional Substack newsletter analytics tool. No signup required. No data stored permanently. Just paste a URL and get:

• Real scraped data — subscriber counts, post engagement (likes, comments), publishing frequency
• AI-powered analysis — GPT-5.2 categorizes your newsletter, estimates metrics, identifies strengths/weaknesses, and suggests growth strategies
• Auto-discovered competitors — AI suggests top competitors in your niche, which you can add to compare side-by-side
• Detailed content analytics — engagement timelines, word count distributions, free vs. paid content mix
• Privacy-first — all data is browser-session scoped. Nothing persists. Hard reset anytime.

---

Features

Track Any Newsletter

Paste any Substack URL (subdomain or custom domain). The scraper handles redirects, custom domains, and Substack's new profile page format automatically.

AI-Powered Insights

Every tracked newsletter gets a full GPT-5.2 analysis:

• Category classification
• Engagement & open rate estimates
• Competitive positioning (Leader / Challenger / Niche Expert / Emerging)
• Strengths, weaknesses, and growth opportunities
• Actionable content strategy tip

Competitor Comparison

• AI suggests top 5 competitors in your niche
• Add any newsletter by URL to your comparison
• Side-by-side charts: subscriber count, radar comparison, detailed metrics table

Content Analytics

• Engagement over time (likes + comments timeline)
• Word count distribution across posts
• Content mix breakdown (free vs. paid)
• Best performing post highlight
• Sortable post table with full metrics

Leaderboard

Rank all tracked newsletters by subscribers, growth rate, open rate, engagement score, or revenue.

Browser-Session Isolation

• Each browser gets its own isolated data via session ID
• Hard Reset button clears everything instantly
• No accounts, no login, no persistent storage

---

Tech Stack

Layer	Technology
Frontend	React 19, Tailwind CSS, Shadcn/UI, Recharts
Backend	FastAPI (Python), Motor (async MongoDB)
Database	MongoDB
Scraping	httpx, BeautifulSoup, lxml (multi-strategy: Archive API, RSS, HTML)
AI	OpenAI GPT-5.2 via emergentintegrations
Design	Swiss High-Contrast (Archetype 4), Chivo font

---

Architecture

Browser (Session ID in localStorage)
    |
    | X-Session-ID header on every request
    |
    v
React Frontend (port 3000)
    |
    | /api/* routes
    |
    v
FastAPI Backend (port 8001)
    |
    |--- Scraper (httpx + BeautifulSoup)
    |       |--- Substack Archive API
    |       |--- RSS Feed fallback
    |       |--- HTML parsing fallback
    |
    |--- AI Analyzer (GPT-5.2)
    |       |--- Category classification
    |       |--- Metric estimation
    |       |--- Competitor suggestion
    |       |--- Insight generation
    |
    v
MongoDB (session-scoped collections)
    |--- accounts (newsletter data)
    |--- posts (scraped posts)
    |--- insights (AI analysis)

---

Getting Started

Prerequisites

• Python 3.11+
• Node.js 18+
• MongoDB 7.0+
• OpenAI API key (or Emergent LLM key)

Installation

# Clone the repo
git clone https://github.com/YOUR_USERNAME/subtracker.git
cd subtracker

# Backend setup
cd backend
pip install -r requirements.txt
cp .env.example .env
# Edit .env with your MongoDB URL and API keys

# Frontend setup
cd ../frontend
yarn install

# Start MongoDB
mongod --dbpath /data/db

# Start backend
cd ../backend
uvicorn server:app --host 0.0.0.0 --port 8001 --reload

# Start frontend (in another terminal)
cd ../frontend
yarn start

Environment Variables

Backend (backend/.env):

MONGO_URL="mongodb://localhost:27017"
DB_NAME="subtracker"
CORS_ORIGINS="*"
EMERGENT_LLM_KEY=your-api-key-here

Frontend (frontend/.env):

REACT_APP_BACKEND_URL=http://localhost:8001

---

API Reference

All endpoints require the X-Session-ID header for data isolation.

Method	Endpoint	Description
POST	/api/track	Track a newsletter by URL. Scrapes data, finds competitors, runs AI analysis.
GET	/api/dashboard	Aggregated stats for all tracked newsletters.
GET	/api/accounts	List all tracked accounts.
GET	/api/accounts/:id	Get account details.
GET	/api/accounts/:id/posts	Get scraped posts for an account.
GET	/api/accounts/:id/insights	Get AI-generated insights.
GET	/api/accounts/:id/competitors	Get competitors in the same category.
GET	/api/comparison?ids=a,b,c	Compare multiple accounts side-by-side.
GET	/api/leaderboard	Ranked list of all tracked accounts.
GET	/api/posts	All posts with optional filtering.
DELETE	/api/accounts/:id	Remove a tracked account.
DELETE	/api/reset	Hard reset - delete all session data.

Example: Track a Newsletter

curl -X POST http://localhost:8001/api/track \
  -H "Content-Type: application/json" \
  -H "X-Session-ID: my-browser-session" \
  -d '{"url": "https://lennysnewsletter.substack.com"}'

Response includes: account data, scraped posts, AI insights (category, strengths, weaknesses, competitors), and auto-discovered competitor accounts.

---

Scraping Strategy

SubTracker uses a multi-strategy approach to handle the full spectrum of Substack newsletter configurations:

1. Substack Archive API (/api/v1/archive) - Primary source. Returns structured JSON with engagement metrics.
2. Custom Domain Detection - When xxx.substack.com redirects, tries www.{slug}.com automatically.
3. RSS Feed Parsing (/feed) - Fallback for newsletters where the API isn't accessible.
4. HTML Scraping - Final fallback for metadata extraction from any page.

Handles:

• *.substack.com subdomain URLs
• substack.com/@username profile URLs
• Custom domains (e.g., www.lennysnewsletter.com)
• Redirects and URL normalization

---

Pages

Page	Description
Dashboard	Hero URL input, KPI cards, growth chart, recent posts
Accounts	Card grid of all tracked newsletters with search & filter
Account Detail	Full analytics with AI insights, posts/competitors/charts tabs
Compare	Manual competitor comparison via URL input, AI suggestions, charts
Analytics	Multi-account growth overlays with area/line charts
Content	Primary account deep-dive: engagement timeline, word count distribution, content mix
Leaderboard	Sortable rankings across 6 metrics with progress bars

---

Design

• Swiss High-Contrast aesthetic with sharp edges (0-4px border radius)
• Chivo font family (300/400/700/900 weights)
• Primary accent: #FF6719 (Substack orange)
• Dark/Light theme toggle with full CSS variable theming
• Data-dense control room grid layout
• No shadows, 1px borders, flat backgrounds
• Micro-animations on hover and page transitions

---

Privacy & Data

SubTracker is designed with privacy as a core principle:

• No accounts or login - completely anonymous usage
• Browser-session scoped - each browser gets isolated data via UUID in localStorage
• Hard Reset - one-click deletion of all data with confirmation dialog
• No tracking cookies - no analytics, no third-party scripts
• Ephemeral by design - data exists only as long as you want it to

---

Contributing

Contributions are welcome! Here's how:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

Ideas for Contributions

• Scheduled re-scraping (auto-refresh data)
• CSV/PDF export for reports
• Bulk URL tracking
• Browser extension for one-click tracking
• Email alerts for subscriber milestones
• Public shareable dashboard links

---

License

This project is licensed under the MIT License. See the LICENSE file for details.

---

Built with Emergent