Future Roadmap

Future Roadmap

Signal's roadmap is organized around two distinct concerns: the intelligence synthesis layer and the supporting infrastructure beneath it.

Intelligence synthesis layer

The core value of Signal is not the pipeline — it is the longitudinal reduction of noise into insight. Each synthesis pass operates at a longer time horizon and produces analysis that is only possible because of the layer below it:

Raw articles   →  Daily brief       (tactical signals)
Daily briefs   →  Weekly report     (emerging patterns)
Weekly reports →  Monthly report    (strategic shifts)      ← Phase 3
Monthly reports → Quarterly report  (structural trends)     ← future horizon

This mirrors how actual intelligence organizations reduce noise. The monthly and quarterly layers are not simply "summaries of summaries" — they enable analysis that is structurally impossible at shorter time horizons: narrative momentum tracking, forecast validation, persistent blindspot detection, and cross-cycle trend emergence.

Supporting infrastructure

Everything else — SonarQube cleanup, source expansion, cross-run deduplication, delivery — supports the synthesis layer but does not define it.

---

Phase 2 — Weekly intelligence summary

Status: ✅ Complete (May 2026). See Weekly-Reports for full documentation.

What was built

Every Sunday at 11:00 PM, a separate launchd job runs the weekly synthesis pipeline (Pass 6). It reads the past 7 days of daily briefs and correlation analyses from signal.db and makes a single LLM call to produce a weekly intelligence brief.

The weekly brief contains:

• Week in Review — the dominant political dynamic of the week as a whole
• Story Arc Tracker — how the top 3–5 stories evolved day by day
• What Escalated — items that grew in significance as the week progressed
• What Was Buried — stories that appeared then disappeared without resolution
• Blindspot of the Week — the most significant story both sides systematically underreported
• Watch List: Next Week — concrete forward-looking items with named entities and deadlines
• Analyst Note — a weekly assessment of trajectory

Key design decisions made

• Read from DB, not HTML files — the structured JSON in correlation_analyses and cluster_analyses is richer than re-parsing HTML. Each day's SITUATION OVERVIEW, WATCH LIST, narrative patterns, and anomalies are extracted and fed to Pass 6.
• Single LLM call — unlike daily Pass 1 (one call per article), the weekly synthesis is one large-context call. Claude handles this well; Ollama would struggle with the context window.
• Gold/amber visual identity — weekly reports use a gold accent scheme (#e3b341) instead of the daily blue, making them immediately distinguishable in the archive and on the landing page.
• Separate weekly_briefs DB table — weekly runs never interfere with daily data.
• Schedule: Sunday 11:00 PM — captures the full week including Sunday morning news, and publishes before Monday morning.

Files added

File	Purpose
pipeline/weekly.py	Pass 6 — reads DB, formats daily summaries, calls LLM
scripts/run_weekly_and_publish.sh	Shell wrapper: runs weekly pipeline + git push
scripts/com.flexrpl.signal.weekly.plist	launchd configuration for Sunday 11 PM

---

Phase 3 — Monthly intelligence summary

Status: Planned (target: late June 2026, once four weekly reports exist as inputs)

Every month, a monthly synthesis pass will read the four weekly briefs and produce a higher-order analysis focused on month-scale trends — story arc completion, what materialized from the watch list, and what dominated the month overall.

Design

The pattern mirrors Phase 2 exactly. A new pipeline/monthly.py module (Pass 7) reads from the weekly_briefs table instead of briefs, condenses each week's brief into key sections, and makes a single LLM call.

Dimension	Daily (Passes 1–5)	Weekly (Pass 6)	Monthly (Pass 7)
Input	RSS articles	7 daily briefs	4 weekly briefs
LLM calls	~150	1	1
DB source	articles, briefs	briefs, correlation_analyses	weekly_briefs
Schedule	Daily 5:00 AM	Sunday 11:00 PM	1st of month, 12:01 AM
Output	brief_YYYYMMDD_HHMM.html	weekly_YYYYWNN_HHMM.html	monthly_YYYY-MM_HHMM.html
Theme accent	Blue #58a6ff	Gold #e3b341	Green #3fb950

Monthly brief sections

These sections are deliberately distinct from the weekly format. Monthly analysis is only meaningful because it can see across four separate weeks simultaneously.

• Month in Review — the dominant political dynamic of the month as a whole
• Narrative Momentum Index — which storylines gained or lost importance across the four weeks; what grew, what faded, what stalled
• Story Arc Completion — which stories that opened in the month resolved, escalated, or quietly disappeared
• Forecast Scorecard — compare each week's watch list against what actually happened the following week; a self-evaluation of the pipeline's predictive accuracy
• Persistent Blindspots — topics that were repeatedly omitted by one side of the spectrum across multiple weeks, revealing structural rather than incidental editorial bias
• Emerging Actors — people and organizations that became increasingly central across the month; signals of rising influence
• Cross-Month Trend Detection — themes that would be invisible in a 24-hour or 7-day window but become clear across 28+ days
• Watch List: Next Month — forward-looking items with named entities and expected timelines
• Analyst Note — monthly trajectory assessment

The Forecast Scorecard is the most structurally novel section — it is the first point in the pipeline where Signal evaluates its own prior analysis rather than just producing new analysis.

Files to add

File	Purpose
pipeline/monthly.py	Pass 7 — reads weekly_briefs, formats weekly summaries, calls LLM
pipeline/prompts.py	Add MONTHLY_BRIEF prompt constant
scripts/run_monthly_and_publish.sh	Shell wrapper: runs monthly pipeline + git push
scripts/com.flexrpl.signal.monthly.plist	launchd configuration for 1st of month, 12:01 AM

Prerequisite

At least four weekly reports must exist in signal.db before the monthly pipeline can run meaningfully. The first full month of weekly data will be available at the end of June 2026.

---

Phase 4 — Quarterly intelligence summary

Status: Future horizon. Not planned for active development until Phase 3 is stable and at least two months of monthly reports exist.

Quarterly is where genuine political trend analysis begins to emerge rather than news analysis. A quarter of data — ~90 daily briefs, ~13 weekly reports, ~3 monthly reports — is long enough to detect structural shifts in the political landscape that are invisible at shorter time horizons.

Unique quarterly analytical outputs would include:

• Structural Shift Detection — which political dynamics fundamentally changed vs. which only appeared to change
• Narrative Lifecycle Analysis — full arc from emergence to resolution or institutionalization for major stories
• Actor Trajectory Index — which people and organizations rose, fell, or consolidated influence over 90 days
• Seasonal Pattern Recognition — distinguishing genuine political shifts from recurring seasonal news cycles
• Forecast Accuracy Review — evaluate monthly Forecast Scorecards against outcomes; a meta-level assessment of intelligence quality

The quarterly layer also enables something the shorter layers cannot: retrospective correction. A monthly event that looked significant at the time can be re-evaluated at 90 days with the benefit of what happened next.

---

Supporting infrastructure

---

Pass 1 parallelization

Status: Deferred. Not necessary for current usage — the ~22 minute daily run completes well before the 5:00 AM window closes.

Current: Pass 1 runs 140–160 Claude CLI subprocess calls sequentially, taking ~22 minutes.

Proposed: Use concurrent.futures.ThreadPoolExecutor to run multiple calls in parallel:

from concurrent.futures import ThreadPoolExecutor, as_completed

with ThreadPoolExecutor(max_workers=10) as executor:
    futures = {
        executor.submit(_llm_call_claude, prompt, timeout): (article, db_id)
        for article, db_id, prompt in work_items
    }
    for future in as_completed(futures):
        article, db_id = futures[future]
        result = future.result()
        # process result...

Expected improvement: 10 workers × ~9 sec/call = Pass 1 completes in ~2–3 minutes instead of ~22. Full run time drops from 22 minutes to ~5–7 minutes.

Considerations:

• Claude API rate limits — Anthropic's API allows high concurrency on Pro/Max plans, but sustained 10-parallel subprocess calls may hit limits. Start with max_workers=5 and tune.
• The Rich progress bar needs updating to work with async completion order.

---

Formal test suite

Status: ✅ Complete (May 2026). See Testing for full documentation.

164 tests across all five pipeline modules. 91% overall coverage. analyzer.py at 100%. All LLM calls and network I/O are mocked — the suite runs in under two seconds.

---

Source expansion

Status: Ongoing as needed

Potential additions to improve spectrum coverage:

Source	Bias	Notes
New York Times	left	Requires RSS subscription
The Atlantic	center-left	Good long-form analysis
The Dispatch	center-right	Conservative anti-Trump perspective
The Intercept	far-left	Investigative/adversarial framing
Just the News	right	Alternative to Breitbart
Substack Politics aggregator	varies	Would require custom parser

---

Cross-run deduplication

Status: Planned

Currently, if an article is published at 11 PM, it may appear in both the Day 1 run (within the 24-hour window) and the Day 2 run (still within the window at 5 AM). URL deduplication is only within-run.

Proposed fix: Before saving articles in store.py, query for URLs already seen in recent runs:

SELECT url FROM articles
WHERE collected_at > datetime('now', '-48 hours')
AND url = ?

This would prevent re-analyzing the same article across consecutive days.

---

Email/push delivery

Status: Idea stage

Instead of only publishing to GitHub Pages, optionally email the daily brief to a subscriber list or push a notification with the report link.

Options:

• Email: Send the HTML report via smtplib or a service like SendGrid
• Push notification: Use a service like Pushover or ntfy.sh
• Slack/Discord webhook: Post a summary and link to a channel

This would be controlled by a new delivery: section in sources.yaml.

---

SonarQube quality gates

Status: In progress

Remaining issues are cognitive complexity warnings in reporter.py and main.py — primarily due to the large HTML template functions. Test coverage is no longer a blocker (91% overall). Key remaining categories:

• Code smells — overly complex functions in reporter.py (_render_story_cards, _render_brief_sections) and _update_index() in main.py
• Duplications — repeated HTML generation patterns across daily and weekly templates