Skip to content

Replace the bill article wall of text with a structured brief - #198

Merged
ThatXliner merged 18 commits into
mainfrom
worktree-bill-brief-articles
Jul 27, 2026
Merged

Replace the bill article wall of text with a structured brief#198
ThatXliner merged 18 commits into
mainfrom
worktree-bill-brief-articles

Conversation

@ThatXliner

@ThatXliner ThatXliner commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

The problem

The article pipeline emits one ~900-word markdown blob per item — four ## sections rendered as-is. The prose is genuinely careful: nonpartisan framing, sponsor claims treated as intent rather than results, removed oversight named rather than called "streamlining."

The problem is structural, not editorial:

  • It's a wall. One decision for the reader: read all of it, or none of it.
  • Everything looks equally important. A $200B authorization, a definition, and a hedge about uncertainty all render as body paragraphs.
  • Provenance is prose-shaped. The article could quote the source, but a quote was just italic text — nothing checked it was real.
  • It can't be reused. A blob can't fill a card, a stat tile, or eventually a video storyboard without re-parsing English.

A reader who bounces off paragraph two leaves with nothing — not even curiosity. That's the Bradbury Principle failing in the most ordinary way.

The brief

Same analysis, stored as typed pieces instead of prose (@acme/validators):

Field What it is
hook One sentence naming the biggest concrete change
facts ≤4 figures — money, deadlines, scope
changes ≤5 provisions as beforeafter
affected ≤4 groups with a direction
unknowns 1–3 things the text does not settle
terms ≤5 glossary entries
sections ≤3 markdown sections — the long version, for people who want to read

The reader can stop after the hook, the tiles, the changes, or go all the way into the source text, and every stopping point is coherent.

Nonpartisan guarantees live in the types

Rather than being trusted to a prompt:

  • kind is mechanical, never evaluativecreates/repeals/expands/restricts/requires/waives/funds/transfers. A brief can say what a provision does; it has no vocabulary for saying whether that's good.
  • before/after splits current law from the change — that blur is where accessible summaries usually go wrong.
  • direction tops out at mixed/unclear — the model can decline to score a group instead of inventing a counterweight to balance the list.
  • unknowns is non-empty — there's no valid brief that claims to have settled everything.
  • Legal status is derived, not generatedderiveLegalStatus() reads the scraped status, so "would" vs "does" comes from a string match rather than an inference.

Argument-level framing deliberately stays in the existing cited dual-lens. Briefs describe mechanism; the lens carries the debate. Keeping them separate means the factual layer can't drift into argument.

Citing original sources

Generation is three steps, only one of which costs a token:

  1. Structure (LLM) — one schema-validated call grounded in the official text plus the existing vetted article.
  2. Verify quotes (deterministic) — every quote is checked against the whole source document, normalizing formatting (smart quotes, hyphenation across line breaks, scraped whitespace) but staying strict about words. Unverified quotes are stripped and the claim is kept — a brief may say less than the model wrote, but it never puts words in a bill's mouth.
  3. Lint framing (deterministic) — loaded vocabulary (common sense, radical, landmark, handout, job-killing, red tape) in the model's own prose triggers one regeneration. Quotes are exempt: a sponsor may call their bill "common sense," and quoting that is reporting.

Rendering

BillBrief.tsx renders each block as the UI element it actually is. Two brand constraints:

  • Nothing color-codes a verdict. A group that "loses" access is drawn exactly like one that "gains" it — direction is an arrow and a word, never green/red. Coloring outcomes reads as an editorial position, and red-vs-green sits one step from red-vs-blue.
  • Every claim keeps a path back to the source. Change cards expose the verbatim provision behind them, one tap away.

Scope

Bills only. The schema is written around legislative mechanics. content_brief already carries a contentType, so adding executive actions or court cases is a generator and a schema pass, not a migration. Content without a brief keeps rendering the markdown article — this is additive, not a cutover.

Verification

  • 6 new unit tests covering quote grounding (accepts wrapped/smart-quoted source text, rejects paraphrase and reordered words), quote stripping, the framing lint and its quote exemption, and status derivation. Full scraper suite: 23 passing.
  • Ran the migration + seed against a throwaway local Postgres, then checked end-to-end: seeded briefs are schema-valid, every seeded quote survives the real verifyBriefQuotes against the bill's stored fullText, the framing lint is clean, and content.getById actually serves the brief through tRPC.
  • Full monorepo typecheck, lint, and format pass. Confirmed the narrow union types flow end-to-end from Drizzle through tRPC into the Expo component.
  • Rendered UI verified on an iPhone 17 Pro simulator (iOS 26.5). The migrated and seeded app was exercised end to end: Bills browse, article overview, structured change cards, affected groups, uncertainty, legislative status, and the original-source CTA all render correctly. Screenshots are attached in the PR conversation below.

Rollout

New and changed bills get briefs automatically. For existing rows:

pnpm --filter @acme/scraper run retroactive-briefs --dry-run
pnpm --filter @acme/scraper run retroactive-briefs --limit 20

pnpm db:seed also writes two example briefs, so the UI is visible locally without an LLM key.

Design writeup: docs/article-generation.md.

Rebased onto #199, which landed the two unrelated bugs found while verifying this (broken seed, silent test discovery).

🤖 Generated with Claude Code

@vercel

vercel Bot commented Jul 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
billion-nextjs Ready Ready Preview, Comment Jul 27, 2026 4:55pm

The article pipeline emitted one ~900-word markdown blob per item. The prose
was careful — nonpartisan framing, sponsor claims treated as intent, removed
oversight named rather than called "streamlining" — but structurally it gave
the reader one decision: read all of it, or none of it. Nothing was scannable,
everything rendered at equal weight, and quotes were just italic text that
nothing verified.

A brief is the same analysis stored as typed pieces instead of prose: a hook,
stat tiles, before/after changes, affected groups, unknowns, a glossary, and
optional long-form sections. The reader can stop at any depth and still have
learned something true.

Editorial guarantees move into the types rather than being trusted to a prompt:

- `kind` on a change is mechanical (creates/repeals/waives/funds), never
  evaluative — a brief can say what a provision does, not whether it's good.
- `before`/`after` forces current law to be stated separately from the change.
- `direction` on an affected group tops out at mixed/unclear, so the model can
  decline to score a group instead of manufacturing symmetry.
- `unknowns` is non-empty: no brief can claim to have settled everything.
- Legal status is derived from the scraped bill status, so "would" vs "does"
  comes from a string match rather than an inference.

Generation is three steps, one of which costs a token. The LLM structures the
brief (grounded in the official text plus the existing vetted article), then
two deterministic passes run: every quote is checked verbatim against the whole
source document and stripped if it doesn't match — the claim survives, the
citation doesn't — and loaded political vocabulary in the model's own voice
triggers one regeneration. Quotes are exempt from the lint: a sponsor may call
their bill "common sense", and quoting that is reporting.

Argument-level framing deliberately stays in the existing cited dual-lens.
Briefs describe mechanism; the lens carries the debate.

Bills only. The schema is written around legislative mechanics, and executive
actions and court cases each need their own design pass — `content_brief`
already carries a contentType, so adding one is a generator, not a migration.
Content without a brief keeps rendering the markdown article.

Two adjacent fixes found while verifying this:
- `pnpm db:seed` had been failing outright: the fixture bill descriptions
  exceed the table's 100-char check constraint. Clamped with the same helper
  the scraper and API use.
- `pnpm test` in the scraper globbed `src/**/*.test.ts` unquoted, so the shell
  expanded it one level deep and silently ran 2 of 5 test files. Quoted, so
  Node's runner does the globbing: 23 tests now discovered instead of 7.

Co-Authored-By: Claude <noreply@anthropic.com>
@ThatXliner

ThatXliner commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator Author

Simulator screenshots

Rendered locally on an iPhone 17 Pro (iOS 26.5) against the migrated and seeded database. This pass restores Dual-Lens near the top, promotes essential definitions, breaks dense analysis into progressive-disclosure cards, and makes cited passages jump to a highlighted match in the in-app original text.

Seeded bills
Browse filtered to seeded bills
Article overview
Article overview and brief toggle
Key terms + Dual-Lens
Key terms promoted above a restored, color-accented Dual-Lens
Change + source action
Stacked before and after states with a View source button
Transit + affected groups
Transit visual and affected-group takeaway cards
Open questions
Numbered open questions following compact affected-group cards
Source deep link
Original text tab scrolled to and highlighting the matching cited passage

The interaction was exercised end to end: View source switches tabs, finds the verified quote in the stored original text, scrolls it into view, and highlights the matching passage. The full official-site exit and Dual-Lens source list remain available so the brief stays a gateway rather than a dead end.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Rich article interaction update

A second simulator pass covers the changes from the latest review:

  • Restored Dual-Lens near the top of the article, with visually distinct but nonpartisan case cards and expandable sources.
  • Promoted essential definitions into Key terms before the analysis.
  • Replaced dense before/after columns with a stacked, color-accented change flow.
  • Reduced affected-group walls of text to a short takeaway with optional context.
  • Added a View source action that switches to Original text, scrolls to the verified quote, and highlights the matching passage.
  • Reframed optional depth around concrete reader questions, such as “Why $200B is not guaranteed.”
Key terms + Dual-Lens
Key terms promoted above a restored, color-accented Dual-Lens
Change + source action
Stacked before and after states with a View source button
Transit + affected groups
Transit visual and affected-group takeaway cards
Open questions
Numbered open questions following compact affected-group cards
Source deep link
Original text tab scrolled to and highlighting the matching cited passage

Verified locally on an iPhone 17 Pro simulator (iOS 26.5) against the refreshed seed data. The full official-site exit and Dual-Lens source list remain available, keeping the generated brief a gateway to primary and further reading rather than a replacement for it.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

@warrensuca thoughts on the UI

@junj-st thoughts on the article design

@yohaann196 @icwtyjj review the backend code

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Final article polish + app-wide detail pass

Pushed in 3d3d563. This pass incorporates the latest review and checks the surrounding app surfaces for the same visual and copy issues:

  • Restores a concise What this means for you lead, keeps Dual-Lens, and moves essential definitions near the top.
  • Uses plain, complete sentences; highlights only useful phrases; and rejects unexplained policy jargon during generation.
  • Adds verified View source deep links, further-reading research, and a readable modal deep dive while keeping the brief a gateway to primary and outside sources.
  • Replaces misleading placeholder photography with deliberate local editorial visuals for infrastructure, transit, and algorithm-transparency stories.
  • Cleans Feed source metadata, removes the duplicate source exit in Original text, and consolidates Elections/Settings guest error states.
Article overview + editorial visual
Infrastructure article with a relevant bridge-repair editorial image and concise opening brief
Feed source details
Feed court-case card with a deliberate algorithm-transparency visual and a single original-source row
Clean ballot error state
Elections screen showing one concise ballot lookup error and a verified polling-place link

Also verified on the iPhone 17 Pro simulator that View source opens Original text at the matching highlighted passage, the Original text tab no longer repeats the bottom source CTA, and guest Settings no longer triggers an unauthorized request.

Validation: Expo typecheck, lint, and formatting; scraper typecheck and 26/26 tests; validators and database typechecks; relevant formatting checks.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Moved Dual Lens below the core article content (after the change, affected-groups, and open-questions sections) and before Keep reading. This keeps the primary explanation uninterrupted while preserving the perspective comparison and follow-on sources. Pushed in ba0a354; Expo typecheck, lint, and formatting pass.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Cited “Why wasn’t this implemented before?” context

Pushed in 25cd9d5.

This adds an expandable historical-context card directly below What this means for you. The collapsed state gives a short answer; expanding it shows documented barriers or earlier attempts, numbered citations beside each claim, and tappable links to the original publishers.

The generation pipeline is now explicitly hybrid:

  1. A bounded agentic loop searches separately for historical context and useful follow-up reading.
  2. It opens and reads at least three promising pages, including at least two for the historical explanation.
  3. The structured writer can cite only URLs that the research loop successfully opened.
  4. Deterministic verification removes invented citations and drops the entire section unless at least two distinct verified sources remain.
  5. Framing and plain-language lint also cover the new historical prose.

The seeded infrastructure and privacy examples now include cited context from Congressional Research Service material, and the brief schema is bumped to v6 so cached rows regenerate.

Validation: local seed refresh; Expo typecheck/lint/format; scraper typecheck and 27/27 tests; validators/database typechecks and formatting.

I did not attach a new simulator capture to this update because the local Simulator accessibility bridge timed out repeatedly during the final visual pass; the component itself compiled and the seeded data path was refreshed successfully.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Updated What would change in aad9487: change cards now render in a snap-style horizontal carousel, leave a visible edge of the next card, and include page dots plus a short swipe cue. This keeps multiple detailed changes from becoming one long vertical wall. I also added a curated data-privacy visual option and applied the existing algorithm-transparency artwork to the seeded privacy example; the generator is instructed to omit images when no bundled visual genuinely fits. Local seed refresh, Expo typecheck/lint/format, scraper typecheck, and all 27/27 tests pass. No new simulator image is attached because the Simulator accessibility bridge is still timing out.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author
03-key-terms-dual-lens 04-change-source 05-affected-groups 06-open-questions 07-source-highlight

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Added backwards compatibility in 354a57d. The API now parses and normalizes shipped v1 and v5 brief records into the current client shape, so existing articles continue rendering instead of disappearing when the schema changes. Missing v1 fields receive safe defaults (affected-group takeaway derived from its existing effect text; empty researched-reading list), while invalid/unknown shapes still fail closed. Separately, the scraper cache now reuses only true v6 records: older records can be served immediately but are regenerated when the scraper encounters them, allowing them to gain cited history, current copy rules, and curated visuals. Validation: API/Expo/database/validator typechecks, API lint/format, scraper typecheck, and 29/29 tests.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Fixed the missing second-card details in a0719ed.

Root cause: change visuals and source quotes are intentionally optional so the generator does not invent evidence or force irrelevant artwork. The seeded second privacy change omitted both fields even though this example supports them.

Updated:

  • added a distinct data-control visual for reviewing and deleting personal data
  • added an exact, grounded Section 3 passage, so card 2 now shows In the text · Sec. 3 and View source
  • expanded the seeded official text so the source action opens and highlights the exact passage
  • updated the generation schema and prompt to evaluate every change independently for a direct quote and relevant visual, instead of effectively enriching only the first card
  • preserved optionality when a source genuinely does not contain a supporting passage or relevant curated image

New data-control visual

Verified locally: refreshed both seeded briefs; Simulator exposes the second image/caption and View source at Sec. 3; Expo typecheck/lint/format pass; scraper typecheck and all 29 tests pass; validator and database checks pass.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Updated the Who it lands on cards in 05b8305.

  • Gains: mint treatment with an up-arrow
  • Loses: coral treatment with a down-arrow
  • Mixed: violet treatment with a minus icon
  • Unclear: amber treatment with a question-mark icon
  • each card now uses a tinted surface, colored outline, icon tile, status chip, and matching disclosure control
  • labels and distinct icons remain in place, so color is not the only signal
  • removed the redundant inner colored rail beside the takeaway after the simulator review; each card now has one clean outline and no double-line treatment

Verified in iPhone 17 Pro Simulator. Expo typecheck, lint, and formatting all pass.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Updated generated-article emphasis and fixed the takeaway clipping in 5564571.

Brief-wide emphasis

  • selective bold spans now render in the hook, before/after copy, affected-group summaries and details, unknowns, glossary definitions, historical context, deep-dive previews/body, and reading recommendations
  • the generation schema and prompt now require one concrete scan target per structured prose field
  • added a deterministic emphasis lint: when a model omits bolding, the pipeline retries with the exact missing fields named
  • bumped the brief generation contract to v7 so cached v6 briefs regenerate, while v1/v5/v6 records remain backwards-compatible and renderable
  • refreshed both local seed briefs to demonstrate the emphasis immediately

Clipping fix

  • the first affected-group takeaway was dropping its second line in React Native even with an explicit newline
  • takeaways now use measured, separately rendered lines that preserve emphasis across line breaks; all three privacy cards show their complete sentences in Simulator
  • kept the cleaned single-outline color treatment with no redundant inner rail

Verified locally

  • iPhone 17 Pro Simulator: full takeaways render without clipping; bold phrases appear in affected groups and What the text does not settle
  • Expo typecheck/lint/format pass
  • scraper typecheck and all 30 tests pass, including new emphasis and v6 compatibility coverage
  • validator and database type/format checks pass; local v7 seed refresh succeeds.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Refined the Who it lands on typography hierarchy in f59cd52.

  • group names now use the strongest bold sans-serif weight as the card header
  • takeaway sentences use a lighter editorial weight and lower-contrast body color
  • only the generated key phrase is bright, bold, and visually dominant
  • Why this matters is reduced to a quieter medium-weight utility label
  • shortened the seeded emphasis spans so bolding highlights a true scan target instead of most of the sentence
  • preserved the measured line rendering from the clipping fix

Verified in iPhone 17 Pro Simulator with refreshed v7 seeds. Full takeaways wrap cleanly and the four text levels are visibly distinct. Expo typecheck/lint/format and database type/format checks pass.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Implemented the Dual-Lens grounding follow-up in 30fa1b1.

What changed:

  • Every newly generated argument must include a cited, concrete precedent or observed result.
  • The research loop now rejects examples that are merely related; it must state how the example demonstrates, supports, or limits the exact argument above it.
  • Citation verification rejects generated examples whose source IDs do not resolve to fetched URLs.
  • Existing cached/legacy lenses remain renderable, while the new contract version forces stale generated lenses to refresh.
  • The UI renders examples as separate, color-matched REAL-WORLD EXAMPLE callouts.

I also corrected the privacy seed so the relationship is explicit:

  • Proponents: California already demonstrates access/deletion rights; the proposal extends similar rights nationwide.
  • Opponents: California has an additional right to limit use of precise-location and genetic data; the proposal excerpt does not expressly preserve that specific protection.

Validation: Expo typecheck + lint, scraper typecheck + 30 tests, DB typecheck, and local seed refresh all pass.

I did not attach the earlier Simulator capture because its copy predates this semantic correction; the final screenshot should be taken from this commit.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Follow-up pushed in 3a58b35: example nuance is now enforced structurally, not left to prompt wording alone.

Each newly generated Dual-Lens example must contain:

  1. Fact — a cited, documented precedent, policy, case, or measured result.
  2. What it shows — a separately validated plain-language explanation of exactly how that fact supports or limits the argument, naming the shared right, mechanism, omission, cost, or tradeoff.

The synthesis prompt explicitly rejects adjacent facts, generic “this is relevant” language, and repeated claims. The UI renders the relevance explanation as its own WHAT IT SHOWS block. Legacy string examples remain readable, and the lens cache contract is bumped so previously generated lenses refresh.

The corrected opponent seed now says:

  • Fact: California lets residents limit businesses’ use of precise-location and genetic data.
  • What it shows: The proposal excerpt creates nationwide access/deletion rights but does not expressly preserve California’s additional limit-use right; that specific gap explains the concern that state protections could be reduced.

Validation remains green: Expo typecheck/lint, scraper typecheck and 30 tests, DB typecheck, and local seed refresh.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Proper text-wrapping fix pushed in ca869e4.

Root cause addressed: the affected-card prose lived under a column child using flex: 1 without an explicit horizontal constraint, with another wrapper around the nested emphasized Text. That allowed intrinsic text measurement to escape the card width on iOS.

The fix:

  • removes the character-count estimator, word packing, onLayout measurement state, and separately rendered lines;
  • gives the affected list, card, body, header, group label, direction chip, and takeaway explicit width/min-width/shrink constraints;
  • renders the takeaway as one normal React Native Text tree, preserving inline bold spans and native word wrapping.

Validation: Expo typecheck and lint pass. Visual confirmation can now use the original long privacy strings without any hand-inserted line breaks.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

The screenshot exposed the remaining root cause; fixed in b5a79b7.

The first takeaway was the only one whose bold phrase had to continue onto another line. EmphasizedText changed from the parent InriaSerif-Regular family to a separately registered InriaSerif-Bold family inside the nested span. On iOS, that long inline run was clipped at the line boundary instead of continuing. The other two cards appeared correct because each bold run fit on one line.

The shared emphasis renderer now keeps the paragraph’s inherited font family and applies fontWeight: 700 to inline emphasis. This preserves one native text layout/wrapping context across the entire paragraph and fixes the issue globally for generated prose—without manual line breaks, character estimates, or seed-specific copy changes.

Expo typecheck and lint pass.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

Correction pushed in ea78241. The prior fontWeight fix wrapped but lost the real custom bold face because these Expo fonts are registered as separate family names.

The final approach restores AlbertSans-Semibold / InriaSerif-Bold. A multi-word emphasized phrase is emitted as real-bold word runs separated by ordinary whitespace, giving iOS native break opportunities between words. React Native still chooses every line; there is no width estimation, character counting, manual line packing, or seed-specific copy change.

This addresses both failures together: long bold phrases can continue onto another line, and the bold styling remains visibly intact. Expo typecheck and lint pass.

@ThatXliner

Copy link
Copy Markdown
Collaborator Author

sorry for all the noise my agent generated. here's the ss:

image image image image image image image image

@ThatXliner
ThatXliner merged commit d51dd55 into main Jul 27, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant