Skip to content

Improve article aggregation coverage and reader guides#2429

Merged
pethers merged 11 commits into
mainfrom
copilot/update-article-generation-content
May 12, 2026
Merged

Improve article aggregation coverage and reader guides#2429
pethers merged 11 commits into
mainfrom
copilot/update-article-generation-content

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 12, 2026

  • Fix Vitest failures introduced by audit-anchor fallback regression
  • Address PR review comments (aggregate.ts, sources-appendix.ts, Article-Generation.md)
  • Fix alias-dedup reporting accuracy in coverage report
  • Constrain supporting-data collector + cap with truncation annotation
  • Compute coverage classification per slot (alias-group-aware)
  • Branch buildSourcesAppendix on used.length for supporting-data-only runs
  • Make readerValueFor derive alias resolution from aliasGroupFor
  • Address follow-up review comments (36043d3)
    • Distinguish alias-suppressed-at-selection from cleaned-to-empty in coverage classification
    • Remove dead EXCLUDED_SUPPORTING_DATA_DIRS / isExcludedSupportingDataPath()
    • Make artifactTitle / artifactIcon alias-aware via aliasLookupKey()
    • Added 3 new vitest cases
  • Regenerate 2026-05-12 articles with corrected icon mappings
  • Address review #4272587528
    • Build Reader Guide from emittedRootSet (post-loop) instead of rootArtifactSet (disk) — prevents broken #anchor links for files cleaned-to-empty; guide spliced at index 1 to preserve Executive Brief → Guide → sections order
    • Fix supportingDataTruncatedCount JSDoc: (and N more not shown)(+N truncated) to match actual rendered output
    • Added vitest: file cleaned-to-empty does NOT appear as a Reader Guide row
  • Full Vitest passes (5468 passed / 30 skipped); npm run lint clean

@github-actions github-actions Bot added documentation Documentation updates testing Test coverage refactor Code refactoring size-l Large change (250-1000 lines) labels May 12, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🏷️ Automatic Labeling Summary

This PR has been automatically labeled based on the files changed and PR metadata.

Applied Labels: documentation,testing,refactor,size-l

Label Categories

  • 🗳️ Content: news, dashboard, visualization, intelligence
  • 💻 Technology: html-css, javascript, workflow, security
  • 📊 Data: cia-data, riksdag-data, data-pipeline, schema
  • 🌍 I18n: i18n, translation, rtl
  • 🔒 ISMS: isms, iso-27001, nist-csf, cis-controls
  • 🏗️ Infrastructure: ci-cd, deployment, performance, monitoring
  • 🔄 Quality: testing, accessibility, documentation, refactor
  • 🤖 AI: agent, skill, agentic-workflow

For more information, see .github/labeler.yml.

Copilot AI requested a review from pethers May 12, 2026 11:13
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers pethers marked this pull request as ready for review May 12, 2026 11:19
Copilot AI review requested due to automatic review settings May 12, 2026 11:19
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR strengthens aggregated analysis/daily/.../article.md outputs by adding an explicit artifact-coverage reconciliation section, linking supporting JSON audit artifacts (instead of inlining them), and fixing Reader Intelligence Guide audit anchors so they always point to an existing audit section (including newer political-classification.md variants).

Changes:

  • Add ## Analysis Artifact Coverage Report plus “Supporting Data Artifacts” links in ## Article Sources, and include supporting .json files in the reported/returned artifact list.
  • Fix Reader Intelligence Guide “Audit appendix” anchor resolution (and HTML aside navigation) to target the strongest available audit section, falling back to article sources when needed.
  • Add filename aliases and ordering updates for stakeholder-impact.md and political-classification.md, with updated docs/tests and regenerated 2026-05-12 articles.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/render-lib.test.ts Updates aggregation-order expectations for new alias entries and appendix tail.
tests/render-lib-leaf-modules.test.ts Adds unit coverage for audit-anchor selection, JSON artifact linking, and the new coverage report.
tests/article-aside.test.ts Updates aside navigation tests for new audit-anchor fallback behavior.
scripts/validate-article.ts Updates Reader Guide row detection to support icon-first table rows.
scripts/render-lib/article-aside.ts Uses auditAnchorForArtifacts() so the audit row links to an existing audit section (or sources fallback).
scripts/render-lib/aggregator/sources-appendix.ts Adds “Supporting Data Artifacts” link section and introduces buildArtifactCoverageReport().
scripts/render-lib/aggregator/reader-guide.ts Introduces auditAnchorForArtifacts() and updates audit-row anchor generation.
scripts/render-lib/aggregator/reader-guide-descriptions-i18n.ts Adds reader-value aliases for stakeholder-impact.md and political-classification.md.
scripts/render-lib/aggregator/order.ts Adds alias groups + ordering entries + titles for new artifact variants.
scripts/render-lib/aggregator/aggregate.ts Collects supporting JSON artifacts, emits coverage report, and extends sources appendix + returned artifact list.
Article-Generation.md Documents JSON artifact handling, updated order, and audit-anchor behavior.
analysis/daily/2026-05-12/realtime-pulse/article.md Regenerated article includes coverage report + supporting JSON links.
analysis/daily/2026-05-12/propositions/executive-brief.md Editorial fix aligning heading to “BLUF”.
analysis/daily/2026-05-12/propositions/article.md Regenerated article includes new coverage report + updated guide + supporting JSON links.
analysis/daily/2026-05-12/motions/executive-brief.md Editorial fix aligning heading to “BLUF”.
analysis/daily/2026-05-12/motions/article.md Regenerated article includes coverage report + supporting JSON links.
analysis/daily/2026-05-12/interpellations/executive-brief.md Editorial refinement to include dok_id identifiers in BLUF text.
analysis/daily/2026-05-12/interpellations/article.md Regenerated article includes coverage report + supporting JSON links.
analysis/daily/2026-05-12/committeeReports/intelligence-assessment.md Minor editorial tweak + trailing whitespace cleanup.
analysis/daily/2026-05-12/committeeReports/article.md Regenerated article includes coverage report + supporting JSON links.

if (file === 'executive-brief.md') return false;
const aliases = aliasGroupFor(file);
if (aliases && [...aliases].some((alias) => emittedRootSet.has(alias))) return false;
return !rootArtifactSet.has(file);
Comment on lines 23 to 37
): string | null {
if (used.length === 0) return null;
if (used.length === 0 && supportingDataArtifacts.length === 0) return null;
const sourceLines = used.map((file) => {
Comment thread Article-Generation.md Outdated

### Canonical political-intelligence order

`AGGREGATION_ORDER` in [`scripts/render-lib/aggregator.ts`](scripts/render-lib/aggregator.ts) publishes sections in this order:
@pethers
Copy link
Copy Markdown
Member

pethers commented May 12, 2026

@copilot apply changes based on the comments in this thread and TypeScript & JavaScript Testing / Unit Tests (Vitest) (pull_request)
TypeScript & JavaScript Testing / Unit Tests (Vitest) (pull_request)Failing after 3m

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 12, 2026

@copilot apply changes based on the comments in this thread and [TypeScript & JavaScript Testing / U...

Addressed in 55ccc99:

  • Vitest failure (audit anchor collision): the reader-guide audit row now falls back to #rm-article-sources (added as the wrapping <section> id), so the substring rm-article-sources-heading only appears once — at the actual h2 — restoring the briefIdx < guideIdx < restIdx < sourcesIdx ordering invariant. Updated the article-aside audit-row test accordingly and loosened the AGGREGATION_ORDER[7] index pin (now asserted relatively) so adding alias filenames doesn't break it.
  • Coverage-report absentOrderedArtifacts: now split into missingFromDisk and presentButFiltered (with alias-group handling against emittedRootSet), and both are reported in the coverage section so the table matches the rendered output.
  • buildSourcesAppendix JSDoc: updated to state null is returned only when both used and supportingDataArtifacts are empty.
  • Article-Generation.md: aggregator links now point at scripts/render-lib/aggregator/ (barrel + order.ts) instead of the non-existent aggregator.ts.

Full Vitest passes locally (5461 passed / 30 skipped), as do npm run lint and validate-article for today's regenerated articles.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

if (file === 'executive-brief.md') return false;
if (!rootArtifactSet.has(file)) return false;
if (emittedRootSet.has(file)) return false;
if (aliasGroupHasEmittedMember(file)) return false;
Comment on lines +86 to 99
'## Analysis Artifact Coverage Report',
'',
'This generated report reconciles the analysis folder with the article projection so reviewers can see what was included, what was linked as supporting data, and which canonical ordered artifacts are not visible in this run (split between "missing on disk" and "present-but-filtered" so the report matches the rendered output).',
'',
'| Coverage area | Count | Reader-facing treatment |',
'|---|---:|---|',
`| Ordered/root markdown sections | ${emittedCount} | Expanded as article sections in the narrative order above |`,
`| Per-document analyses | ${perDocCount} | Expanded under \`## Per-document intelligence\` immediately after significance scoring |`,
`| Supporting data artifacts | ${dataCount} | Linked in Article Sources, not expanded inline |`,
'',
`**Absent canonical ordered artifacts (missing from disk)**: ${fmt(input.absentOrderedArtifacts)}`,
'',
`**Present-but-filtered canonical artifacts (on disk but omitted by alias de-duplication or empty-after-cleaning)**: ${fmt(input.presentButFilteredArtifacts ?? [])}`,
].join('\n');
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.

Comment on lines +177 to +196
const collectSupportingDataArtifacts = (): string[] => {
const out: string[] = [];
const walk = (dir: string, prefix: string): void => {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })
.sort((a, b) => a.name.localeCompare(b.name))) {
const full = path.join(dir, entry.name);
const rel = prefix ? `${prefix}/${entry.name}` : entry.name;
if (entry.isDirectory()) {
// Pass-1 drafts are superseded by the Pass-2 audited artifacts and
// must not be surfaced as reader-facing supporting data.
if (isExcludedSupportingDataPath(rel)) continue;
walk(full, rel);
} else if (/\.json$/i.test(entry.name)) {
out.push(rel);
}
}
};
walk(subfolderAbsPath, '');
return out;
};
Comment on lines +230 to +264
// 1. missingFromDisk — canonical artifact filename never appeared in
// the subfolder (the analysis run never produced it).
// 2. presentButFiltered — the file existed on disk but was trimmed
// to an empty body by `cleanArtifactBody()` or skipped via the
// alias-de-duplication, so the article projection omits it.
// Reporting both keeps the coverage table honest about what the
// reader actually sees vs. what raw analysis output exists.
const aliasGroupHasEmittedMember = (file: string): boolean => {
const aliases = aliasGroupFor(file);
if (!aliases) return false;
return [...aliases].some((alias) => emittedRootSet.has(alias));
};
const missingFromDisk = AGGREGATION_ORDER.filter((file) => {
if (file === 'executive-brief.md') return false;
if (aliasGroupHasEmittedMember(file)) return false;
return !rootArtifactSet.has(file);
});
// Files present on disk but skipped because another alias-group member
// was already emitted. Tracked separately so the report is accurate:
// these are not "absent" (the file exists) but they are not emitted
// either — the canonical alias wins, and only that alias is visible.
const aliasDedupedArtifacts = AGGREGATION_ORDER.filter((file) => {
if (file === 'executive-brief.md') return false;
if (!rootArtifactSet.has(file)) return false;
if (emittedRootSet.has(file)) return false;
return aliasGroupHasEmittedMember(file);
});
// Files present on disk but trimmed to an empty body by cleanArtifactBody().
const presentButFiltered = AGGREGATION_ORDER.filter((file) => {
if (file === 'executive-brief.md') return false;
if (!rootArtifactSet.has(file)) return false;
if (emittedRootSet.has(file)) return false;
if (aliasGroupHasEmittedMember(file)) return false;
return true;
});
Comment on lines +242 to +265
const missingFromDisk = AGGREGATION_ORDER.filter((file) => {
if (file === 'executive-brief.md') return false;
if (aliasGroupHasEmittedMember(file)) return false;
return !rootArtifactSet.has(file);
});
// Files present on disk but skipped because another alias-group member
// was already emitted. Tracked separately so the report is accurate:
// these are not "absent" (the file exists) but they are not emitted
// either — the canonical alias wins, and only that alias is visible.
const aliasDedupedArtifacts = AGGREGATION_ORDER.filter((file) => {
if (file === 'executive-brief.md') return false;
if (!rootArtifactSet.has(file)) return false;
if (emittedRootSet.has(file)) return false;
return aliasGroupHasEmittedMember(file);
});
// Files present on disk but trimmed to an empty body by cleanArtifactBody().
const presentButFiltered = AGGREGATION_ORDER.filter((file) => {
if (file === 'executive-brief.md') return false;
if (!rootArtifactSet.has(file)) return false;
if (emittedRootSet.has(file)) return false;
if (aliasGroupHasEmittedMember(file)) return false;
return true;
});
const absentOrderedArtifacts = missingFromDisk;
Comment on lines +41 to +47
const lines = [
'## Article Sources',
'',
'Each section above projects one analysis artifact. The full audited markdown is available on GitHub:',
'',
...sourceLines,
];
Comment on lines +574 to +582
const aliasBaseName =
baseName === 'stakeholder-impact.md'
? 'stakeholder-perspectives.md'
: baseName === 'political-classification.md'
? 'classification-results.md'
: baseName;
return READER_VALUE_I18N[file]?.[lang]
?? READER_VALUE_I18N[baseName]?.[lang]
?? READER_VALUE_I18N[aliasBaseName]?.[lang];
@pethers
Copy link
Copy Markdown
Member

pethers commented May 12, 2026

@copilot apply changes based on the comments in this thread

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 12, 2026

@copilot apply changes based on the comments in this thread

All five comments addressed in 36043d3:

  1. Supporting-data collector scope + cap (aggregate.ts:177-196): now constrained to top-level *.json and documents/*.json only — no arbitrary recursion. Hard cap MAX_SUPPORTING_DATA_ARTIFACTS = 100 with truncation count surfaced in the coverage report as (+N truncated).
  2. Alias-group double-counting (aggregate.ts:230-264): coverage classification now runs per-slot (alias-aware), with each alias group rendered as a.md / b.md / c.md so a missing slot is reported once, not N times.
  3. Sources-appendix when used is empty (sources-appendix.ts:41-47): buildSourcesAppendix now branches on used.length — canonical "Each section above projects one analysis artifact…" prose only when sections exist; supporting-data-only runs get a dedicated preamble.
  4. Hard-coded alias lookup (reader-guide-descriptions-i18n.ts:574-582): readerValueFor now derives alias resolution from aliasGroupFor (single source of truth in FILENAME_ALIASES); new alias filenames automatically inherit their group's description.

Three new vitest cases added covering the new buckets, supporting-data truncation annotation, and supporting-data-only preamble. Full vitest passes (5464 passed / 30 skipped) and lint is clean.

@pethers pethers requested a review from Copilot May 12, 2026 12:51
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

scripts/render-lib/aggregator/order.ts:112

  • political-classification.md is now part of AGGREGATION_ORDER and an alias of classification-results.md, but it isn’t currently covered by the artifact UI i18n/icon maps (ARTIFACT_TITLE_I18N / ARTIFACT_ICON in scripts/political-intelligence/i18n/artifact-i18n.ts). This means localized HTML navigation and the sources card grid may fall back to an English prettified title and a generic icon. Add explicit i18n/icon entries for this filename (or make lookups consult FILENAME_ALIASES) so both audit filenames render consistently across languages.
  // ─── Phase G — Critique & alt hypotheses ───────────────────────────
  'devils-advocate.md',
  // ─── Phase H — Audit appendix ──────────────────────────────────────
  'classification-results.md',
  'political-classification.md',
  'cross-reference-map.md',
  'horizon-pir-rollforward.md', // long-horizon supplementary
  'methodology-reflection.md',
  'data-download-manifest.md',

Comment on lines +283 to +298
for (const slot of orderedSlots) {
const onDisk = slot.members.filter((m) => rootArtifactSet.has(m));
const emitted = slot.members.filter((m) => emittedRootSet.has(m));
if (onDisk.length === 0) {
missingFromDisk.push(slot.label);
continue;
}
if (emitted.length === 0) {
// None of the on-disk members survived cleaning.
presentButFiltered.push(onDisk.join(', '));
continue;
}
// Slot satisfied; surface alias siblings that were skipped.
const skipped = onDisk.filter((m) => !emittedRootSet.has(m));
if (skipped.length > 0) aliasDedupedArtifacts.push(...skipped);
}
Comment on lines 53 to 61
const EXCLUDED_SUPPORTING_DATA_DIRS = new Set(['pass1']);

function isExcludedSupportingDataPath(rel: string): boolean {
for (const dir of EXCLUDED_SUPPORTING_DATA_DIRS) {
if (rel === dir || rel.startsWith(`${dir}/`)) return true;
}
return false;
}

Comment on lines 66 to 72
// Stakeholder lens, parliamentary arithmetic (who can pass it), and
// voter exposure (whose interests are at stake) — clustered so the
// "WHO" question is answered as one block.
'stakeholder-perspectives.md',
'stakeholder-impact.md',
'coalition-mathematics.md',
'voter-segmentation.md',
@pethers
Copy link
Copy Markdown
Member

pethers commented May 12, 2026

@copilot apply changes based on the comments in this thread

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 12, 2026

@copilot apply changes based on the comments in this thread

All three comments addressed in f515bbc:

  1. aggregate.ts:283-298 (alias-suppressed vs cleaned-empty): the loop now records aliasSuppressedAtSelection (only when the file actually exists on disk) at the continue site, and readSection records cleanedToEmpty when cleanArtifactBody() trims the body to empty. Per-slot reconciliation buckets non-emitted on-disk siblings by why they weren't emitted, so a sibling that was cleaned to empty no longer mis-reports as alias-de-duped.
  2. aggregate.ts:53-61 (dead code): EXCLUDED_SUPPORTING_DATA_DIRS / isExcludedSupportingDataPath() removed — the collector doesn't recurse so it could never match.
  3. order.ts:66-72 (alias-unaware i18n): artifactTitle / artifactIcon in political-intelligence/i18n/artifact-i18n.ts are now alias-aware via a new aliasLookupKey() helper that walks the alias group from FILENAME_ALIASES. Verified end-to-end in the regenerated 2026-05-12 articles: stakeholder-impact.md → 👥 and political-classification.md → 🏷️ (instead of the generic 📝 fallback).

3 new vitest cases added; full vitest passes (5467 passed / 30 skipped) and lint is clean.

@pethers pethers requested a review from Copilot May 12, 2026 13:10
Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/eef7a52b-aa6b-4b69-b990-27a32ce86cc1

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.

Comment on lines 220 to 222
readSection('executive-brief.md', false);
sections.push(buildReaderGuide(rootArtifactSet, docsExist));

* Number of supporting-data JSON artifacts that exist on disk but
* were truncated from `supportingDataArtifacts` because the
* collector hit its hard cap. Surfaced in the report as an explicit
* `(and N more not shown)` annotation. Optional; defaults to 0.
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers
Copy link
Copy Markdown
Member

pethers commented May 12, 2026

@copilot apply changes based on the comments in this thread

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 12, 2026

@copilot apply changes based on the comments in this thread

Both issues addressed in a061fd4:

  1. aggregate.ts:220-222buildReaderGuide is now called after all emission loops, using emittedRootSet instead of rootArtifactSet. It's spliced into sections at index 1 so the reading order (Executive Brief → Reader Guide → sections) is preserved. A file that cleanArtifactBody() trims to empty will no longer produce a broken #anchor in the guide. New vitest case added to cover this regression.

  2. sources-appendix.ts:85 — JSDoc wording updated from (and N more not shown) to (+N truncated) to match the actual buildArtifactCoverageReport() output.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers pethers merged commit d773a43 into main May 12, 2026
18 checks passed
@pethers pethers deleted the copilot/update-article-generation-content branch May 12, 2026 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Documentation updates refactor Code refactoring size-l Large change (250-1000 lines) testing Test coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants