RFC: Source-Check Badge Integration Across TableBase Tables

[OAGr]

Context

We have a mature source-check system (LLM-verified structured data against source URLs) with 2,439 verdicts in the DB. Currently, 9 UI locations show SourceCheckBadge indicators. The goal is to surface verification status across all TableBase tables — organization tabs, directory pages, detail pages.

The naive plan ("add a Verified column to every table") has serious problems. This RFC lays out a revised approach based on red-teaming, UX analysis, coverage data, and architectural review.

Problem: Coverage Is Wildly Uneven

Before designing the UI, the actual verdict coverage must inform the approach:

Record Type	Verdicts	Total Records	Coverage
division	91	112	81%
publication	143	186	77%
personnel	411	550	75%
policy-stakeholder	80	~100	~80%
investment	9	72	13%
grant	261	5,904	4%
funding-program	0	61	0%
funding-round	0	91	0%
equity-position	0	13	0%
benchmark-result	0	~200	0%

FactBase facts: 1,444 verdicts across ~3,000+ facts (~48%).

Key insight: Adding a "Verified" column to a table where 95% of rows are blank is anti-information — it teaches users to ignore the column and implies everything is unverified. A dedicated column only makes sense at >50% coverage.

Rejected Approach: Per-Row Column Everywhere

The original plan proposed adding a SourceCheckBadge column to ~16 tables across 3 phases. Problems:

1. Empty columns are misleading. A "Verified" header with mostly-blank cells implies comprehensive checking that doesn't exist. Users learn to ignore the column.
2. Tables are already wide. AI models has 6+ columns. Adding a 7th that's usually empty wastes space.
3. Inconsistent patterns. Current integrations use two different approaches — dedicated column (divisions, funding-programs) vs inline next to name (equity, career-history). Scaling without unifying makes the UX incoherent.
4. Phase 3 is not real. Resources/announcements/press don't map to existing record types and verifying them is a fundamentally different problem (an announcement IS the source).

Revised Approach: Three-Tier Display System

Tier 1: Inline Dots (Universal)

Replace the current text pill badge ("Verified", "Disputed") with a compact colored dot (4-6px circle) rendered inline next to the entity name in the first column of any table. Tooltip on hover shows the full verdict + confidence.

● Anthropic Safety Fund       (green dot = confirmed)
● MIRI Research Grant         (red dot = contradicted)
  Some Other Grant            (no dot = unchecked)

Why this works:

• No extra column — zero width cost
• Unchecked rows show nothing — zero visual noise
• Red/amber dots catch the eye for problems without cluttering happy paths
• Already proven: equity-section and career-history use an inline pattern today

Component: Create SourceCheckDot (~15 lines) alongside existing SourceCheckBadge. Standardize all current inline usages to this pattern.

Apply to: Every table, regardless of coverage level. Since unchecked rows show nothing, there's no "empty column" problem.

Tier 2: Section Summary Banner (High-Coverage Tables)

For tables with >30% verdict coverage, add a slim summary bar above the table:

🛡 12 of 47 records source-checked · 10 confirmed · 1 outdated · 1 disputed

Why this works:

• Communicates coverage honestly — "12 of 47" tells users exactly how much is verified
• Uses getRecordVerdictStats() which already exists but has zero consumers
• One component, reusable across all tables
• Degrades gracefully — if zero records are checked, banner doesn't render

Component: VerificationSummaryBanner (~30 lines). Server component that calls getRecordVerdictStats(recordType).

Apply to: divisions, publications, personnel, policy-stakeholders (>70% coverage), and eventually grants/investments as coverage grows.

Tier 3: Dedicated Column (Only for >70% Coverage)

Keep the existing dedicated "Verified" column pattern only for tables where most rows will show a badge. Currently that means: divisions, publications, personnel.

Tables that currently have a dedicated column but low coverage (funding-rounds at 0%) should remove the column and switch to Tier 1 dots.

Coverage-Gated Rollout

Instead of phasing by "how easy is it to wire up," phase by data readiness:

Phase 1: High-Coverage Types (divisions, publications, personnel, policy-stakeholders)

• These already have >70% coverage
• Add inline dots + summary banners
• Keep or add dedicated columns where they exist
• Convert any inconsistent inline patterns to the new SourceCheckDot
• Tables affected: divisions directory, divisions section (org page), publications directory, publications section (org page), people directory (new), people tab (org page, already has career-history), legislation stakeholder table (already integrated)

Phase 2: Run Source-Check Orchestrator for Uncovered Types

• Run pnpm crux source-check-orchestrate targeting funding-programs, funding-rounds, equity-positions, benchmark-results
• Set a coverage threshold: don't add UI indicators until a type reaches >30%
• This is pipeline work, not UI work

Phase 3: Medium-Coverage Types (grants, investments, + newly checked types)

• Once coverage reaches >30%, add inline dots + summary banners
• Grants (5,904 records) will take significant orchestrator time/budget to reach 30%
• Investments (72 records) could be fully checked quickly

Phase 4: New Record Types (if warranted)

• Resources/press/announcements are a different verification problem — the resource IS the source
• AI model pricing/specs could use benchmark-result record type for scores, but pricing verification needs a new ai-model record type
• Don't force these into the existing system. Evaluate whether a new data model is needed.

Architecture: Keep It Simple

The four architectural approaches evaluated:

Approach	Verdict
A. Copy-paste per table	✅ Recommended (with small utility)
B. Verdict-aware wrapper	Marginal gain (~5 LOC/table saved)
C. Build-time enrichment	Couples verdicts to data build; violates YAML-is-truth
D. Context provider	Incompatible with server components
E. DOM overlay	Anti-pattern in React; breaks SSR

The real variation is in rendering, not data loading. Each table positions badges differently (inline, column, after source link). No abstraction can unify this. The getRecordVerdict() call is one line — the per-table cost is ~15 lines of straightforward code.

One small utility worth adding in tablebase.ts:

function enrichWithVerdicts<T extends { key: string | number }>(
  rows: T[], recordType: string
): (T & { verdict: RecordVerdict | null })[] {
  return rows.map(r => ({ ...r, verdict: getRecordVerdict(recordType, String(r.key)) }));
}

This standardizes the Pattern 2 (server-to-client) enrichment and prevents record type string typos.

Immediate Action Items

1. Create SourceCheckDot component — compact dot variant of SourceCheckBadge
2. Create VerificationSummaryBanner component — section-level coverage summary
3. Add enrichWithVerdicts() utility to tablebase.ts
4. Reconcile record type mappings — collectionToRecordType (8 types) is missing 5 types from VALID_RECORD_TYPES (publication, benchmark-result, entity-event, entity-assessment, secondary-market-price)
5. Remove "Verified" column from zero-coverage tables (funding-rounds currently has it)
6. Wire Tier 1 dots to high-coverage tables (divisions, publications, personnel, policy-stakeholders)
7. Run source-check orchestrator for funding-programs, funding-rounds, equity-positions to build coverage
8. Call getRecordVerdictStats() — it exists with zero consumers. Wire it to the summary banner.

Open Questions

• Should the SourceCheckDot have an onClick that opens a detail popover (showing sources checked, last verified date, confidence)?
• Should we add a "Data Quality" aggregate indicator to entity page headers (e.g., "37/52 records verified for Anthropic")?
• What's the budget/timeline for running the orchestrator to reach 30% coverage on the zero-coverage types?
• Should stale verdicts (>6 months old) show differently? The lastComputedAt field exists but is never displayed.

[OAGr]

On questions:

1. Add as a later phase. Good but less critical.
2. "Should we add a "Data Quality" aggregate indicator to entity page headers " -> ALso do as later phase.
3. "What's the budget/timeline for running the orchestrator to reach 30% coverage on the zero-coverage types?" -> $100 is fine. $1k is tougher. Think through.
4. "Should stale verdicts (>6 months old) show differently? " -> Seems good. Maybe a late phase.

[OAGr]

Decisions on Open Questions

1. SourceCheckDot onClick detail popover → Later phase

Good idea but not critical for initial rollout. The tooltip already shows verdict + confidence. A richer popover (sources checked, last verified date, link to evidence) can come after the dot + banner system is established and people are actually using it.

2. Entity-level "Data Quality" aggregate indicator → Later phase

An entity header like "37/52 records verified for Anthropic" would be valuable but depends on the per-section infrastructure being in place first. Build Tier 1 (dots) and Tier 2 (section banners) first, then aggregate section banners into an entity-level indicator as a natural extension.

3. Orchestrator budget to reach 30% coverage → ~$16 total, easily under $100

At $0.01/verification (Claude Haiku 4.5), the costs are trivial:

Type	Records	Need for 30%	Already Have	Gap	Cost
funding-program	61	19	0	19	$0.19
funding-round	91	28	0	28	$0.28
equity-position	13	4	0	4	$0.04
benchmark-result	~200	60	0	60	$0.60
investment	72	22	9	13	$0.13
grant	5,904	1,772	261	1,511	$15.11

Total to 30% on all types: ~$16. Grants dominate the cost (5,904 records).

Going further:

• 70% coverage on all types: ~$55-60 (grants = ~$41 of that)
• Full coverage on everything: ~$60 (grants 5,904 × $0.01 = $59 ceiling)

This is well within a $100 budget. The constraint isn't money — it's time (each verification involves an HTTP fetch of the source URL + LLM call) and source URL availability (records without a source/sourceUrl field can't be checked at all, which may cap actual achievable coverage below 100%).

Recommendation: Run pnpm crux source-check-orchestrate --budget=50 --type=record to check all record types. This would get small types to near-100% coverage and grants to ~30%+. Then evaluate whether the remaining unchecked grants simply lack source URLs.

4. Stale verdict display (>6 months) → Later phase

Makes sense. lastComputedAt is already in the RecordVerdict interface but never displayed. Options:

• Fade the dot opacity for verdicts older than 6 months
• Add a clock icon or "⏳" suffix to the tooltip
• Mark them needsRecheck: true in the data layer (field already exists)

Not critical for initial rollout since the system is relatively new and most verdicts are recent. Build it when the first verdicts actually hit 6 months old.

---

Updated phasing with these decisions:

1. Phase 1: SourceCheckDot + VerificationSummaryBanner components, wire to high-coverage types (divisions, publications, personnel, policy-stakeholders)
2. Phase 2: Run orchestrator (~$50 budget) to build coverage on zero-coverage types
3. Phase 3: Wire dots + banners to newly-covered types (grants, investments, funding-rounds, etc.)
4. Phase 4 (later): Detail popover on dot click, entity-level aggregate indicator, stale verdict styling

[OAGr]

Superseded by Discussion #3993 (Source-Check → Sourcing Rename — Implementation Plan).

The architecture and naming decisions in this discussion have been consolidated into the 7-phase sourcing rename plan. Key changes: routes become /sourcing, tables rename to record_sources/source_verdicts, components standardize on SourcingDot/SourcingBadge.

Phase 0: #4001 | Phase 1: #4002

[OAGr]

Branch: claude/deploy-tasks-verify-cmd

Closing as superseded — already marked superseded by #3993. Canonical UI dots plan lives in #3802. — Discussion review 2026-04-08.