Skip to content

Fix dashboard CSV loading: use absolute /cia-data/ URLs so /dashboards/* pages stop 404'ing local CSVs#2403

Merged
pethers merged 2 commits into
mainfrom
copilot/investigate-dashboard-setup
May 11, 2026
Merged

Fix dashboard CSV loading: use absolute /cia-data/ URLs so /dashboards/* pages stop 404'ing local CSVs#2403
pethers merged 2 commits into
mainfrom
copilot/investigate-dashboard-setup

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 11, 2026

🔎 Investigation

The task asked three questions:

  1. Are dashboard pages correctly loading D3/Chart.js and CSVs? Does Vite build & S3 deploy ship the right artifacts?
    D3, Chart.js and PapaParse are bundled via register-globals in cia-entry.ts / main.ts and load fine.
    vite build compiles every src/browser/dashboards/*.ts module, and postbuild (in package.json) copies cia-data/ into dist/cia-data/.
    s3-deploy.yml (scripts/deploy-s3.sh) uploads dist/** and aws s3 sync auto-detects text/csv for .csv files. ✅
  2. Is cia-data correctly available where the loaders expect?
    It is — at /cia-data/... on both GitHub Pages and S3.
  3. Do all dashboards load the correct values?
    They do eventually, but only via the GitHub-raw fallback because eleven loaders built relative cia-data/… URLs. From /dashboards/parties.html those resolved to /dashboards/cia-data/… → 404, so the local CSV (and the SW cache) was bypassed on every visit.

✅ Fix

Switch every dashboard loader to absolute, root-relative /cia-data/… URLs so the local-first hop succeeds from any document depth (/, /dashboards/<slug>.html, /dashboard/index*.html, /politician-dashboard.html).

Loaders updated

Module Before After
dashboards/stats-loader.ts cia-data/extraction_summary_report.csv /cia-data/extraction_summary_report.csv
dashboards/party-dashboard.ts cia-data/party/${file} /cia-data/party/${file}
dashboards/election-cycle.ts cia-data/election-cycle/… × 4 /cia-data/election-cycle/… × 4
dashboards/committees-dashboard.ts cia-data/… × 5 /cia-data/… × 5
dashboards/coalition-dashboard.ts cia-data/… × 7 /cia-data/… × 7
dashboards/seasonal-patterns.ts cia-data/seasonal/… /cia-data/seasonal/…
dashboards/pre-election.ts cia-data/pre-election/… × 2 /cia-data/pre-election/… × 2
dashboards/anomaly-detection.ts cia-data/seasonal/… /cia-data/seasonal/…
dashboards/ministry-dashboard.ts cia-data/ministry/ (config base) /cia-data/ministry/
dashboards/risk-dashboard.ts cia-data/politician/… /cia-data/politician/…
dashboards/politician-dashboard.ts LOCAL_DATA_BASE = 'cia-data' LOCAL_DATA_BASE = '/cia-data'
cia/data-loader.ts csvBaseURL = '../cia-data/' csvBaseURL = '/cia-data/'

coalition-loader.ts already used /cia-data/, so no change needed.

🧪 Tests

  • New tests/dashboard-cia-data-absolute-paths.test.ts — strips comments from each dashboard module's source and asserts no remaining bare cia-data/… (or '../cia-data/') string literal.
  • Updated tests/cia-data-loader-orchestrator.test.ts — expects the new /cia-data/foo.csv base URL (the one test that pinned the old '../cia-data/' prefix).
  • Full Vitest suite: 102 files / 5425 tests passing.
  • tsc -p tsconfig.browser.json --noEmit: clean.
  • Existing self-referential string-literal tests in tests/coalition-dashboard.test.js & tests/committees-dashboard.test.js are unaffected (they verify hand-rolled arrays, not the dashboard config).

📈 Expected impact

  • Dashboards now hit the local S3-served CSV on first request → no GitHub-raw round-trip for the 12 modules above.
  • Service-worker cache (/\/cia-data\/.*\.csv$/i) starts hitting on dashboard pages; previously the SW's runtime cache kept missing because the request URL contained /dashboards/cia-data/… and the origin returned 404 before the SW could store anything useful.
  • LCP/TTI improvement on /dashboards/* pages (no fallback waterfall).
  • Resilience to GitHub raw-URL throttling / outages on the dashboard hub.

🔒 Security & compliance

Pure URL string-prefix change in client-side fetch() URLs (relative → root-absolute). No new sinks/sources, no auth/crypto/SSRF surface. CodeQL declared trivial; Code Review passed with no comments.

Copilot AI requested a review from pethers May 11, 2026 08:19
@github-actions github-actions Bot added testing Test coverage size-m Medium change (50-250 lines) labels May 11, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🏷️ Automatic Labeling Summary

This PR has been automatically labeled based on the files changed and PR metadata.

Applied Labels: testing,size-m

Label Categories

  • 🗳️ Content: news, dashboard, visualization, intelligence
  • 💻 Technology: html-css, javascript, workflow, security
  • 📊 Data: cia-data, riksdag-data, data-pipeline, schema
  • 🌍 I18n: i18n, translation, rtl
  • 🔒 ISMS: isms, iso-27001, nist-csf, cis-controls
  • 🏗️ Infrastructure: ci-cd, deployment, performance, monitoring
  • 🔄 Quality: testing, accessibility, documentation, refactor
  • 🤖 AI: agent, skill, agentic-workflow

For more information, see .github/labeler.yml.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers pethers marked this pull request as ready for review May 11, 2026 08:22
Copilot AI review requested due to automatic review settings May 11, 2026 08:22
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes dashboard CSV loading for pages under /dashboards/* by switching local-first CSV URLs from relative cia-data/... paths (which resolve to /dashboards/cia-data/... and 404) to root-relative /cia-data/... paths, aligning with the deployed artifact location in dist/cia-data/.

Changes:

  • Updated multiple dashboard loaders to fetch local CSVs via absolute root-relative /cia-data/... URLs.
  • Updated CIADataLoader’s csvBaseURL to /cia-data/ to make orchestrated loads depth-independent.
  • Added/updated Vitest coverage to prevent regressions (new “no relative cia-data literals” test + updated orchestrator expectation).

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/dashboard-cia-data-absolute-paths.test.ts New regression test scanning key modules to ensure no relative cia-data/... (or ../cia-data/...) string literals remain.
tests/cia-data-loader-orchestrator.test.ts Updates expectation to match the new /cia-data/foo.csv base URL behavior.
src/browser/dashboards/stats-loader.ts Switches stats hero local CSV to /cia-data/extraction_summary_report.csv so it works from any page depth.
src/browser/dashboards/seasonal-patterns.ts Updates seasonal local CSV URL to /cia-data/... for /dashboards/seasonal-patterns.html.
src/browser/dashboards/risk-dashboard.ts Updates risk dashboard local CSV URL to /cia-data/... for /dashboards/risk.html.
src/browser/dashboards/pre-election.ts Updates pre-election local CSV URLs to /cia-data/... for /dashboards/pre-election.html.
src/browser/dashboards/politician-dashboard.ts Sets LOCAL_DATA_BASE to /cia-data so local-first URLs work from any document depth.
src/browser/dashboards/party-dashboard.ts Updates party dashboard local fetch URL to /cia-data/party/${filename} for /dashboards/parties.html.
src/browser/dashboards/ministry-dashboard.ts Updates ministry dashboard local base URL to /cia-data/ministry/ (preserving trailing slash behavior).
src/browser/dashboards/election-cycle.ts Updates election-cycle local CSV URLs to /cia-data/... for /dashboards/election-cycle.html.
src/browser/dashboards/committees-dashboard.ts Updates committees dashboard local CSV URLs to /cia-data/... for /dashboards/committees.html.
src/browser/dashboards/coalition-dashboard.ts Updates coalition dashboard local CSV URLs to /cia-data/... for /dashboards/coalitions.html.
src/browser/dashboards/anomaly-detection.ts Updates anomaly-detection local CSV URL to /cia-data/... for /dashboards/anomaly-detection.html.
src/browser/cia/data-loader.ts Changes csvBaseURL to /cia-data/ so orchestrated loads aren’t dependent on the importing page’s directory depth.

@pethers pethers merged commit a1c1ab8 into main May 11, 2026
30 checks passed
@pethers pethers deleted the copilot/investigate-dashboard-setup branch May 11, 2026 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size-m Medium change (50-250 lines) testing Test coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants