Rename source-check → sourcing: Architecture + Migration Plan

[OAGr]

Overview

The current "source-check" system is the primary infrastructure for tracking where structured data comes from and whether it's accurate. But it has naming problems, architectural inconsistencies, and redundant mechanisms that make it confusing.

This discussion proposes:

1. Rename "source-check" → "sourcing" to reflect that it covers both add-time and post-hoc verification
2. Rename the core tables to clarify their role as the primary source-record relationship
3. Standardize the source column across ~20 tables
4. Remove the redundant sourceResourceId FK columns
5. Keep entity_resources as a separate system (entity-level, not record-level)

Why "Sourcing" Instead of "Source-Check"

"Source-check" implies a post-hoc verification step. But this system also handles:

• Source addition at data-entry time
• LLM discovery of new sources
• Inline verification during sync operations
• Source quality assessment

"Sourcing" covers all of these naturally — "we're sourcing this grant" works for both adding and verifying sources.

Precedent: Wikidata calls these "references" (attached to statements). W3C PROV uses "Entity/Activity/Agent." ClaimReview separates "Claim" from "ClaimReview." Our system is closest to ClaimReview's model — records are claims, sourcing evidence is the review.

Table Renames

Current	Proposed	Why
source_check_evidence	record_sources	This IS the source-record relationship, not a "check"
source_check_verdicts	source_verdicts	Aggregate verdict on sourcing quality
record.source (text)	Keep as-is (see below)

Why keep source as the column name?

After analysis, renaming source → import_source on ~20 tables is high cost for marginal benefit. The source column is well-understood: "URL where this data came from." The real fix is ensuring record_sources (the many-to-many table) is the authoritative source list, with the inline source column being a convenience/seed value.

Why remove sourceResourceId?

The sourceResourceId FK on 5 tables (personnel, grants, funding_rounds, investments, equity_positions) is redundant with record_sources.resourceId. The evidence table already links records to resources with richer metadata (verdict, confidence, extracted quotes). Maintaining both creates drift risk.

Schema After Migration

record (grant, personnel, etc.)
  └── source: text URL                    ← seed/convenience URL (unchanged)
  └── (no sourceResourceId)               ← removed, use record_sources instead

record_sources (was source_check_evidence)
  └── recordType + recordId               ← identifies any record
  └── resourceId (FK → resources)          ← linked to resource catalog
  └── sourceUrl                            ← URL checked
  └── verdict, confidence                  ← per-source assessment
  └── extractedValue, extractedQuote       ← evidence
  └── discoveryMethod (NEW)                ← "import", "manual", "llm_discovery"
  └── isPrimary (existing)                 ← marks canonical source
  └── fieldName (nullable)                 ← cell-level: NULL=whole row, "amount"=specific field
  └── Multiple rows per record

source_verdicts (was source_check_verdicts)
  └── recordType + recordId + fieldName    ← aggregate across all sources
  └── verdict, confidence                  ← overall assessment
  └── needsRecheck, nextCheckDue           ← staleness tracking

entity_resources (UNCHANGED - separate system)
  └── entityId + resourceId                ← entity-level, not record-level
  └── authoredByEntity, isSubject          ← relationship flags

Key design decisions

1. isPrimary stays as explicit boolean — deriving from highest confidence is fragile (shifts silently on re-check)
2. discoveryMethod added — single text column tracking how the source-record link was created ("import", "manual", "llm_discovery", "source_check")
3. Cell-level fieldName kept — deterministic matcher uses it for amount/date verification. Default NULL = whole row.
4. record_sources keyed on (recordType, recordId, COALESCE(resourceId, sourceUrl)) — resource-based dedup when available, URL-based fallback
5. Negative results stored — verdict="unverifiable" rows prevent re-checking the same dead URL
6. entity_resources stays separate — different concept (entity relationships vs record provenance)

Migration Plan

~8-10 PRs over 6-10 weeks:

Phase	What	Files	Risk
0	Lint guard (prevent new source-check in code)	~5	None
1	DB table rename (ALTER TABLE RENAME)	~15-20	Low
2	API dual-mount /api/sourcing + server file rename	~10-12	Low
3	Frontend pages + components	~35-40	Low
4	CLI + crux directory rename	~40-45	Medium (conflicts)
5	GitHub Actions + docs	~15-20	None
6	Drop sourceResourceId from 5 tables	~10-15	Medium
7	Remove aliases (after 4-8 weeks)	~10	Low

Key migration notes:

• ALTER TABLE RENAME is instant (metadata-only, no table lock)
• Drizzle will generate DROP+CREATE — must manually write RENAME migrations
• API aliases maintained during transition, removed in Phase 7
• Next.js redirects from /source-checks → /sourcing for SEO

Open Questions

1. Should we rename the source column on records to import_source? The plan above keeps it as-is. Renaming ~20 tables is high-cost. But the current name implies authoritativeness it doesn't have.
2. Should discoveryMethod be added to record_sources? Low cost but increases schema surface.
3. What should the URL routes be? /sourcing/grant/123 vs /sources/grant/123 vs /evidence/grant/123
4. Should we rename SourceCheckDot → SourcingDot or VerificationDot? The dot component is used in ~22 files.

Tasks

• Finish March 2026 rename: kill verification residue + remove sourceResourceId #4001 — Finish March 2026 rename: kill verification residue + remove sourceResourceId

• Rename Drizzle symbols: sourceCheckEvidence → recordSources, sourceCheckVerdicts → sourceVerdicts #4002 — Rename Drizzle symbols: sourceCheckEvidence → recordSources, sourceCheckVerdicts → sourceVerdicts

[OAGr]

Superseded Work

The following issues have been closed as superseded by this plan:

• Rename /source-checks to /records — unified record detail page #3923 (Rename /source-checks → /records) — route will be /sourcing, not /records
• Run sourceResourceId backfill + literature seed after entity-resources PRs merge #3974 (Run sourceResourceId backfill + seed) — sourceResourceId is being removed in Phase 0 (Finish March 2026 rename: kill verification residue + remove sourceResourceId #4001)
• Unify verdict vocabularies: source-check vs citation pipeline #3447 (Unify verdict vocabularies) — absorbed into Phase 1 (Rename Drizzle symbols: sourceCheckEvidence → recordSources, sourceCheckVerdicts → sourceVerdicts #4002)

Merged PRs that added sourceResourceId infrastructure (#3977, #3978, #3969) will be partially reverted in Phase 0.

Still-valid issues that will need terminology updates after the rename lands: #3444, #3900, #3548, #3609, #3546.