Political Data Roadmap: From Skeleton to Definitive Reference (6-Month Plan)

[OAGr]

Political Data Roadmap: From Skeleton to Definitive Reference (6-Month Plan)

Context

The political data feature shipped in early April 2026 as an MVP: 57+ politician entities with qualitative positions, empty PG tables (political_scores, political_offices, political_races), an empty /races page, and a /politicians directory with UI bugs. The infrastructure is mature (tables, CLI commands, API routes, 40+ documented external sources in political-tracking-resources.yaml) but almost nothing is populated with real data.

Previous ambitious plans (#3129, #3156) designed great architectures but closed before data was actually ingested. This plan prioritizes the opposite order: data first, infrastructure only when needed.

Guiding Principle

Resources → TableBase → FactBase → WikiBase

1. First, connect to external data sources and ingest raw structured data
2. Then, populate PG tables with clean, normalized records
3. Then, create FactBase entries for temporal/provenance-tracked claims
4. Finally, generate wiki pages from rich structured data (not the other way around)

---

Current State (Baseline)

Asset	Status	Gap
Politician entities	57+ YAML, positions-only	No office records, no scores, no career data, no wiki pages
/politicians directory	Renders, has UI bugs (#3621, #3636)	Party/status inference broken, name spacing
/races page	Empty (#3628)	Zero races populated
/legislation directory	38 policies	60% stakeholders unlinked (#3484), 20/38 have zero stakeholders (#3494)
political_scores table	Schema exists	0 rows
political_offices table	Schema exists	0 rows
political_races + race_candidates	Schema exists	~8 seed rows, stale
External data sources	40+ documented in YAML	0 connected, 0 ingesting
Auto-update sources	4 political web-search feeds	News only, no structured data feeds
Person detail pages	Positions table renders	Sources hidden, no legislation links, empty estimates/confidence

---

Month 1: Data Source Connections & First Bulk Ingestion

Goal: Have real data flowing from 3+ external sources into PG tables. Fix the most embarrassing rendering gaps.

1.1 Fix Rendering Gaps (Week 1)

• Fix Politicians directory: inferParty and inferStatus have gaps — 7 missing party, Newsom misclassified #3621 (party/status inference) and Politicians table: name and district text run together without spacing #3636 (name spacing) on /politicians
• Render top-level sources array on person detail pages (the 7 URLs for Alex Bores etc. that are invisible)
• Add website link prominently on politician profiles (donation/campaign page)
• Cross-link politicians to legislation where they appear as stakeholders (P1: 60% of legislation stakeholders have no entity link (86/144 unlinked) #3484 partial fix)
• Hide empty Estimate/Confidence columns when no politician has those fields populated

1.2 Interest Group Scorecards — First Bulk Data (Weeks 2-3)

This is the highest-value, lowest-effort data source. Scorecards are structured, public, and directly EA-relevant.

Priority sources (by EA relevance from political-tracking-resources.yaml):

1. Humane World Action Fund — animal protection scorecard (very-high EA relevance)
2. LCV (League of Conservation Voters) — environmental scorecard (published annually, CSV-friendly)
3. Council for a Livable World — nuclear risk ratings (high EA relevance)
4. Foreign Policy for America — foreign policy scorecard
5. ACLU — civil liberties
6. Heritage Action — conservative baseline

Deliverables:

• Build crux tb political scores ingest --source=<id> command
• Ingest 2024 + 2025 scorecards for all 6 sources (~500+ score records per source)
• Render scorecard data on politician detail pages (radar chart or table by issue area)
• Add scorecard summary columns to /politicians directory

1.3 Office Data Population (Week 3-4)

• Run crux tb political seed-offices (already exists) to populate from YAML descriptions
• Supplement with GovTrack/Congress.gov bulk data: current term, committee assignments
• Add state legislators from Open States API
• Render office history on politician detail pages (replacing the inferred party/status)

1.4 Data Source Registration

• Register each connected source in data_sources table with accessMethod, updateFrequency
• Create first source_snapshots for each ingestion (versioned, auditable)

Month 1 exit criteria: political_scores has 2,000+ rows from 6 scorecard sources. political_offices has records for all 57+ politicians. Person detail pages show sources, office info, and scorecards.

---

Month 2: Campaign Finance & Legislative Data

Goal: Integrate the two most important political data domains — money and votes.

2.1 Campaign Finance — OpenSecrets + FEC (Weeks 5-7)

• Build FEC bulk data importer: candidate fundraising totals, PAC contributions
• Create campaign_finance PG table (or extend existing tables): candidate, cycle, raised, spent, cash_on_hand, pac_contributions
• Import OpenSecrets industry/sector contribution breakdowns for tracked politicians
• Track AI-specific PACs: Leading the Future, Think Big, American Mission, Public First Action
• Build crux tb political finance ingest --cycle=2026 command
• Render campaign finance on politician detail pages (fundraising bar chart, top donors/sectors)
• Add finance columns to /politicians directory (total raised, AI PAC support)

2.2 Voting Records — Congress.gov + GovTrack (Weeks 7-8)

• Build Congress.gov API integration for roll call votes on tracked legislation
• Create political_votes PG table: politician, bill, vote (yea/nay/abstain/not_voting), date, chamber
• Import votes for all legislation entities already in the wiki (~38 policies)
• Render individual vote records on legislation pages (Individual legislator roll call votes on legislation pages #2756 — previously closed as too ambitious)
• Cross-link: politician page shows "Voted Yea on SB 1047", legislation page shows vote breakdown

2.3 Legislation Stakeholder Resolution (Week 8)

• Automated entity resolution for the 86 unlinked stakeholders (P1: 60% of legislation stakeholders have no entity link (86/144 unlinked) #3484)
• Build crux tb political resolve-stakeholders using fuzzy name matching against entity table
• Manual review pass for ambiguous matches
• Target: >90% stakeholder linkage (up from 40%)

Month 2 exit criteria: Campaign finance data for all tracked politicians. Vote records for all tracked legislation. Stakeholder linkage >90%.

---

Month 3: Races, Predictions & State-Level Expansion

Goal: /races goes from empty to the definitive AI-election tracker. Expand beyond federal to state.

3.1 Race Discovery & Population (Weeks 9-10)

• Comprehensive audit of 2026 races for AI relevance (target: 40+ races)
• Populate political_races with real data: federal House/Senate, governors, key state legislature
• Import candidate data into race_candidates with AI stances, PAC support
• Build /races page into a filterable, sortable directory (by state, level, AI relevance)

3.2 Prediction Market Integration (Weeks 10-11)

• Build Polymarket API connector for election markets
• Build Metaculus API connector for election questions
• Create prediction_snapshots table: market, question_id, probability, volume, timestamp
• Auto-sync prediction data daily via cron/auto-update
• Render probability trends on race detail pages (sparkline charts)
• Add "Win probability" column to /races directory

3.3 State-Level Legislation Tracking (Weeks 11-12)

• Integrate LegiScan API for state AI bills (1,500+ bills across 45 states)
• Build crux tb political bills ingest --source=legiscan command
• Create legislation_tracking PG table (or extend existing): bill_id, state, status, last_action, ai_relevance_score
• Auto-classify AI relevance using LLM triage on bill titles/summaries
• Render state bill tracker dashboard (map visualization or table by state)
• Link state bills to relevant politician entities (sponsors, co-sponsors)

3.4 Politician Wiki Pages — Top 10 (Week 12)

• Use crux w create pipeline to generate wiki pages for the 10 most important politicians
• Pages draw from TableBase data (scores, offices, finance, votes) — not manually written
• Priority: Alex Bores, Scott Wiener, Ted Lieu, Chuck Schumer, Richard Blumenthal, Todd Young, Valerie Foushee, Marsha Blackburn, Josh Hawley, Maria Cantwell

Month 3 exit criteria: /races has 40+ real races with candidates and prediction probabilities. State bill tracker covers all 50 states. 10 politician wiki pages live.

---

Month 4: Automated Freshness & FactBase Integration

Goal: Political data stays current without manual intervention. FactBase provides temporal provenance.

4.1 Auto-Update Political Data Feeds (Weeks 13-14)

• Convert scorecard ingestion to scheduled auto-update (annual refresh per source)
• Add FEC filing auto-sync (quarterly FEC deadlines: Apr 15, Jul 15, Oct 15, Jan 31)
• Add prediction market daily snapshots via auto-update cron
• Add Congress.gov vote sync (triggered by legislation status changes)
• Add LegiScan state bill status sync (weekly)
• Monitor via /internal/auto-update-runs dashboard

4.2 FactBase Political Facts (Weeks 14-16)

• Create FactBase entities for key politicians (career-history, employed-by, board-seats equivalents for political roles)
• Temporal facts: "Alex Bores holds office NY Assembly District 73" with validFrom/validTo
• Position facts with provenance: "Alex Bores supports AI safety regulation" citing RAISE Act authorship
• Funding facts: "Alex Bores received $X from Leading the Future PAC in 2026"
• Build crux fb political sync to generate FactBase YAML from PG table data

4.3 Source-Checking Political Claims (Weeks 15-16)

• Run crux fb source-check on political FactBase facts
• Verify politician positions against cited sources (are the scorecard numbers accurate? do the vote records match?)
• Flag stale claims (politician changed position, lost election, etc.)
• Render source-check status on politician pages (green/yellow/red verification badges)

Month 4 exit criteria: All political data auto-refreshes on schedule. FactBase has temporal facts for top 30 politicians. Source-check coverage >80%.

---

Month 5: Rich Analysis & Cross-Cutting Views

Goal: The wiki becomes analytically useful, not just a data directory.

5.1 Committee & Caucus Tracking (Weeks 17-18)

• Import congressional committee membership data (GovTrack/Congress.gov)
• Create committee_memberships PG table or FactBase relations
• Track key committees: Senate Commerce, House Energy & Commerce, Senate AI Caucus, etc.
• Render committee pages showing all members with their AI scores and positions
• "Who controls AI policy?" analysis page showing committee composition and likely votes

5.2 PAC Influence Analysis (Weeks 18-19)

• Cross-reference PAC spending → candidate → voting record
• Build analysis: "Do AI PAC-supported candidates vote differently on AI bills?"
• Correlation dashboards: spending vs. win rate, spending vs. AI stance
• Track dark money through 501(c)(4) orgs where possible (OpenSecrets data)

5.3 Politician Comparison Tools (Week 19-20)

• Side-by-side comparison for any 2-4 politicians (scorecard radar charts, voting overlap, funding sources)
• "Who agrees with whom?" clustering based on AI-relevant votes
• Export/embed functionality for researchers and journalists

5.4 Wiki Pages — Expand to 30+ Politicians (Week 20)

• Generate wiki pages for all politicians with sufficient TableBase data
• Pages auto-include: office history, scorecard summary, key votes, funding, prediction markets
• Interlink with legislation pages, organization pages, race pages

Month 5 exit criteria: Committee tracking live. PAC influence analysis published. 30+ politician wiki pages. Comparison tools functional.

---

Month 6: Election Coverage & Public Launch

Goal: Ship the definitive AI-election reference before the 2026 primaries heat up.

6.1 Election Dashboard (Weeks 21-22)

• Unified election dashboard: all tracked races, prediction trends, spending, key dates
• Calendar view of upcoming primaries, filing deadlines, debate dates
• Alert system: notify when prediction markets move >5%, new FEC filings, bill status changes
• Embeddable widgets for race predictions and scorecard summaries

6.2 Data Quality & Completeness Audit (Week 23)

• Audit all data sources for coverage gaps and staleness
• Verify entity linkage completeness (target: >95% of all cross-references resolve)
• Source-check audit on all published political claims
• Performance audit: page load times for data-heavy politician/race pages

6.3 Public-Facing Polish (Weeks 23-24)

• SEO for politician names, race names, legislation names
• Open Graph/social cards for sharing race predictions
• RSS feed for political data updates
• API documentation for external consumers of political data
• "How to use this data" guide for EA researchers, journalists, policy advocates

6.4 Retrospective & Roadmap Update

• What worked, what didn't
• Data source ROI analysis (which sources were most valuable per hour of integration work?)
• Plan for general election coverage (Nov 2026)
• Identify new domains to apply this resource→TableBase→FactBase→WikiBase pattern

Month 6 exit criteria: Election dashboard live. All data auto-refreshing. 40+ races, 50+ politician pages, 6+ scorecard sources, campaign finance, vote records, prediction markets. Published and externally referenceable.

---

Success Metrics (6-Month Targets)

Metric	Baseline (Apr 2026)	Target (Oct 2026)
political_scores rows	0	5,000+
political_offices rows	0	200+
political_races rows	~8 (stale)	50+
race_candidates rows	~12 (stale)	150+
Politician wiki pages	0	50+
External data sources connected	0	10+
Auto-refreshing data feeds	0	8+
Legislation stakeholder linkage	40%	>95%
Scorecard sources ingested	0	6+
Campaign finance records	0	500+
Vote records	0	2,000+
Prediction market snapshots	0	10,000+

---

Key Risks

Risk	Mitigation
Data source APIs change/break	Snapshot-based ingestion with validation; alert on fetch failures
Scope creep into non-EA-relevant races	Strict AI/EA relevance filter on all ingestion; manual review queue
Stale data creates trust issues	Auto-update with staleness alerts; source-check pipeline catches drift
Legal concerns with scraping	Prefer official APIs (FEC, Congress.gov, LegiScan); respect robots.txt
Election cycle timing pressure	Month 1-3 deliverables are independently valuable even without months 4-6
Previous plans (#3156, #3129) stalled	This plan is data-first, not infrastructure-first; each month produces visible output

---

Relationship to Previous Plans

• Political Races: 2-Month Research Plan for 2026 AI-Relevant Elections #3156 (Political Races 2-Month Plan): Month 3 of this plan subsumes and completes the race discovery and prediction market phases. Key difference: we populate data sources first (months 1-2) before building race pages.
• Political Stability & AI: Wiki Augmentation Plan #3129 (Political Stability & AI): The entity creation and wiki page portions are handled by months 4-5. The FactBase integration is handled in month 4.
• P1: 60% of legislation stakeholders have no entity link (86/144 unlinked) #3484 (Stakeholder linking): Resolved in month 2 as part of legislative data integration.
• P2: Publications missing authors, legislation missing provisions/stakeholders #3494 (Missing provisions/stakeholders): Addressed progressively through months 2-3 as legislative data is enriched.

[OAGr]

Revised Strategy (April 3): Discussion-First, Not Issue-First

The original strategy proposed decomposing discussions into issues, then executing issues. After testing this (decomposed 7 discussions → 22 issues, implemented 6), the conclusion is: this adds unnecessary overhead for big-picture work.

What actually works

Discussions are projects. Each one describes a coherent chunk of value with phases and context. The right model is:

1. One agent session binds to one discussion and works through it over hours
2. Context compounds — by phase 3, the agent deeply understands the system
3. Issues are created along the way when the agent hits blockers or discovers subtasks — not upfront
4. The discussion body tracks progress via a machine-readable metadata block

Agent-project metadata

Discussions that are agent-workable get a metadata block:

<!-- agent-project
priority: high | medium | low
status: not-started | in-progress | blocked | done
phases_total: N
phases_done: N
last_agent_session: YYYY-MM-DD
blocker: (optional free text)
-->

This lets crux commands rank discussions and lets agents know where to pick up.

The loop

Instead of: pick issue → implement → reset → pick issue

It's: pick discussion → build context → work through phases → ship PRs → update discussion → continue or hand off

A sustained session on one discussion produces better work than 5 isolated issue sessions touching the same domain.

Skill: /work-on-discussion N

New skill that: loads the discussion, checks what's done, implements the next phase, ships a PR, updates the discussion, and continues. See .claude/commands/work-on-discussion.md.

When issues still matter

• Small concrete bugs — don't need a discussion, just fix them
• Tasks discovered during discussion work — agent creates issues for things it can't do now
• External contributions — issues are the entry point for outside collaborators
• The crux gh issues next queue — still useful for quick wins between project sessions