Add Statskontoret data integration support by Copilot · Pull Request #1993 · Hack23/riksdagsmonitor

Copilot · 2026-04-25T08:36:37Z

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/dc62517a-f53c-423f-b327-3d2856b258f8 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

github-actions · 2026-04-25T08:37:15Z

🏷️ Automatic Labeling Summary

This PR has been automatically labeled based on the files changed and PR metadata.

Applied Labels: documentation,dependencies,testing,refactor,size-xl

Label Categories

🗳️ Content: news, dashboard, visualization, intelligence
💻 Technology: html-css, javascript, workflow, security
📊 Data: cia-data, riksdag-data, data-pipeline, schema
🌍 I18n: i18n, translation, rtl
🔒 ISMS: isms, iso-27001, nist-csf, cis-controls
🏗️ Infrastructure: ci-cd, deployment, performance, monitoring
🔄 Quality: testing, accessibility, documentation, refactor
🤖 AI: agent, skill, agentic-workflow

For more information, see .github/labeler.yml.

github-actions · 2026-04-25T08:37:44Z

🔍 Lighthouse Performance Audit

Category	Score	Status
Performance	85/100	🟡
Accessibility	95/100	🟢
Best Practices	90/100	🟢
SEO	95/100	🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/6d0707ac-bee8-4c66-b550-cd699273652c Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

github-actions · 2026-04-25T08:51:01Z

🔍 Lighthouse Performance Audit

Category	Score	Status
Performance	85/100	🟡
Accessibility	95/100	🟢
Best Practices	90/100	🟢
SEO	95/100	🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/6d0707ac-bee8-4c66-b550-cd699273652c Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

github-actions · 2026-04-25T08:52:55Z

🔍 Lighthouse Performance Audit

Category	Score	Status
Performance	85/100	🟡
Accessibility	95/100	🟢
Best Practices	90/100	🟢
SEO	95/100	🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/f3c03785-61ec-4ef8-acf8-c12c952c7ded Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

github-actions · 2026-04-25T10:08:08Z

🔍 Lighthouse Performance Audit

Category	Score	Status
Performance	85/100	🟡
Accessibility	95/100	🟢
Best Practices	90/100	🟢
SEO	95/100	🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copilot · 2026-04-25T10:09:46Z

@copilot apply changes based on the comments in this thread

Addressed in b064b08:

Densified sparse worksheet rows via Array.from({ length: row.length }, (_, i) => row[i] ?? '') so cells with explicit refs that skip columns no longer leave holes that misalign rowsToRecords.
Removed the unused 'page' member from StatskontoretResourceType (the classifier returns 'unknown' for plain HTML pages).
Aligned analysis/statskontoret/README.md to Public / High Integrity / Medium-High Availability to match THREAT_MODEL.md and SECURITY_ARCHITECTURE.md.
Added a Vitest case covering sparse-row densification.

20/20 targeted tests pass, ESLint clean, build:lib green, parallel_validation green (Code Review: 0 comments, CodeQL: 0 alerts).

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/e7e16b7d-265b-46d9-8628-73a43c059298 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

github-actions · 2026-04-25T10:22:36Z

🔍 Lighthouse Performance Audit

Category	Score	Status
Performance	85/100	🟡
Accessibility	95/100	🟢
Best Practices	90/100	🟢
SEO	95/100	🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/e7e16b7d-265b-46d9-8628-73a43c059298 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

github-actions · 2026-04-25T10:25:57Z

🔍 Lighthouse Performance Audit

Category	Score	Status
Performance	85/100	🟡
Accessibility	95/100	🟢
Best Practices	90/100	🟢
SEO	95/100	🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

…ation Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/7f40b8bb-7e1e-4b19-9881-f1ad5386d8f0 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

github-actions · 2026-04-25T10:37:28Z

🔍 Lighthouse Performance Audit

Category	Score	Status
Performance	85/100	🟡
Accessibility	95/100	🟢
Best Practices	90/100	🟢
SEO	95/100	🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

…, fix test comment Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/7f40b8bb-7e1e-4b19-9881-f1ad5386d8f0 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

github-actions · 2026-04-25T10:41:09Z

🔍 Lighthouse Performance Audit

Category	Score	Status
Performance	85/100	🟡
Accessibility	95/100	🟢
Best Practices	90/100	🟢
SEO	95/100	🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

github-actions · 2026-04-25T10:42:50Z

🔍 Lighthouse Performance Audit

Category	Score	Status
Performance	85/100	🟡
Accessibility	95/100	🟢
Best Practices	90/100	🟢
SEO	95/100	🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copilot

Pull request overview

Adds first-class Statskontoret open-data integration (client + CLI + persistence + docs/tests) to complement existing IMF/SCB/World Bank provider tiers.

Changes:

Introduces scripts/statskontoret-client.ts (discovery, XLSX/ZIP parsing, headcount + budget outturn helpers, typed errors) and scripts/statskontoret-fetch.ts CLI wrapper for workflows.
Extends persistence layer with persistStatskontoretData() and adds jszip dependency for workbook/ZIP parsing.
Adds comprehensive Vitest coverage and a new analysis/statskontoret/ documentation hub (inventory, data dictionary, use cases) plus architecture/threat-model updates.

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/statskontoret-inventory.test.ts	Validates the new Statskontoret inventory JSON matches the in-code source catalogue and decision matrix.
tests/statskontoret-fetch.test.ts	Covers CLI argument parsing + guards (including allowlist enforcement and typed CLI errors).
tests/statskontoret-client.test.ts	Covers discovery parsing, XLSX/ZIP parsing, aggregation helpers, and typed error surfaces.
scripts/statskontoret-fetch.ts	Adds workflow-friendly CLI to discover/download/parse Statskontoret artifacts with optional persistence.
scripts/statskontoret-client.ts	Adds the core Statskontoret client, parsers, aggregators, allowlist guard, and typed error type.
scripts/parliamentary-data/data-persistence.ts	Adds `statskontoret` documentType + `persistStatskontoretData()` for cache/provenance sidecars.
package.json	Adds `jszip` for XLSX/ZIP parsing.
package-lock.json	Locks `jszip` and transitive dependencies.
analysis/statskontoret/use-cases.md	Documents canonical use cases (headcount, budget execution, outturn summaries).
analysis/statskontoret/indicators-inventory.json	Introduces a machine-readable Statskontoret dataset inventory + provider decision matrix.
analysis/statskontoret/data-dictionary.md	Documents field mappings, inference rules, freshness discipline, and persistence layout.
analysis/statskontoret/README.md	Adds the Statskontoret integration hub and CLI quick reference.
THREAT_MODEL.md	Adds STRIDE threat assessment section for Statskontoret ingestion.
TESTING.md	Documents Statskontoret test suites and expectations (no live network).
SECURITY_ARCHITECTURE.md	Adds security architecture notes/controls for the new integration.
README.md	Adds top-level documentation entry for Statskontoret integration and quick commands.
MINDMAP.md	Adds current-state mindmap branch for Statskontoret integration.
FLOWCHART.md	Adds current-state data-flow diagram for Statskontoret ingestion/persistence.
DATA_MODEL.md	Adds data-model extension notes for Statskontoret entities and persisted artifacts.
ARCHITECTURE.md	Adds current-architecture section describing how Statskontoret fits into the provider tiering and workflows.

+
+  private async fetchWithTimeout(url: string): Promise<Response> {
+    const resolved = resolveStatskontoretUrl(url, this.baseURL);
+    assertStatskontoretFetchTarget(resolved);


+      if (!response.ok) {
+        throw new StatskontoretError(`Statskontoret API error: ${response.status} ${response.statusText} for ${response.url}`, 'http');
+      }
+      return response;


+ * typed rows sorted by year ascending, then month ascending (NaN last for
+ * annual rows), then documentType alphabetically.  For single-type workbooks
+ * (e.g. a file explicitly downloaded as "Inkomst"), pass
+ * `options.documentType` to set the label uniformly.


+    "version": "1.1",
+    "description": "Machine-readable inventory of Statskontoret open datasets used by Riksdagsmonitor for Swedish government-body and central-government budget context. Complements IMF (primary economic projections), SCB (Swedish official statistics), World Bank (non-economic global context), and Riksdag/Regering data.",
+    "lastUpdated": "2026-04-25",
+    "effectiveDate": "2026-04-25",
+    "source": "Statskontoret open data (www.statskontoret.se)",
+    "classification": "Public",
+    "clients": {
+        "cli": "tsx scripts/statskontoret-fetch.ts (commands: list-sources, discover, headcount, budget-outturn)",
+        "library": "scripts/statskontoret-client.ts (StatskontoretClient class)",
+        "persistence": "scripts/parliamentary-data/data-persistence.ts (persistStatskontoretData)"
+    },
+    "notes": {
+        "firewallAllowlist": "www.statskontoret.se",
+        "noMcp": "Statskontoret is not an MCP server. Agentic workflows invoke the TypeScript CLI via the bash tool, mirroring IMF's no-MCP client pattern.",
+        "formats": "Myndighetsförteckningen is published as Excel. Årsutfall and Månadsutfall expose both Excel and CSV ZIP downloads. Budget time-series pages link to annual official-statistics publications and related open-data tables.",
+        "privacy": "Public authority/agency data and aggregate budget data only; no private-person data. Authority names and agency-level budget lines are public administrative records."
+    },
+    "datasets": {
+        "myndighetsforteckning": {
+            "title": "Myndighetsförteckning – öppna data",
+            "url": "https://www.statskontoret.se/analys-och-statistik/oppna-data/myndighetsforteckning/",
+            "cadence": "Annual snapshot; source page metadata observed as last-modified 2026-02-06 for the 2025 workbook.",
+            "coverage": "Summerande statistik 2025; tidsserier 2007–2025; förteckning 2025; förteckning 2007–2025.",
+            "format": [
+                "xlsx"
+            ],
+            "primaryUse": "Headcount of government bodies, grouped by department, leadership form and special organs; department headcount over time from 2007 onward.",
+            "keyFields": [
+                "År",
+                "Myndighet",
+                "Departement / departementstillhörighet",
+                "Årsarbetskrafter",
+                "Ledningsform",
+                "Särskilda organ"
+            ],
+            "derivedArtifacts": [
+                {
+                    "id": "headcount-by-department",
+                    "description": "Sum årsarbetskrafter by year and department, with authority count per group.",
+                    "script": "tsx scripts/statskontoret-fetch.ts headcount --url <xlsx-url> --persist",
+                    "storage": "analysis/data/statskontoret/myndighetsforteckning/headcount-by-department.json"
+                }
+            ],
+            "committees": [
+                "KU",
+                "FiU",
+                "AU"
+            ],
+            "admiralty": "A1"
+        },
+        "budget-time-series": {
+            "title": "Tidsserier, statens budget m.m.",
+            "url": "https://www.statskontoret.se/analys-och-statistik/officiell-statistik/tidsserier-statens-budget-m.m",
+            "cadence": "Annual official statistics release.",
+            "coverage": "Final outcomes for central-government revenue, expenditure, balance and related public-finance tables, generally from 1995.",
+            "format": [
+                "html-publication",
+                "linked-open-data"
+            ],
+            "primaryUse": "Long-run Swedish central-government budget context for finance, tax and public-administration analysis.",
+            "committees": [
+                "FiU",
+                "SkU",
+                "KU"
+            ],
+            "admiralty": "A1",
+            "derivedArtifacts": [
+                {
+                    "id": "budget-outturn",
+                    "description": "Long-run central-government budget time series (revenue and expenditure) from 1995 onward parsed into StatskontoretBudgetRow objects.",
+                    "script": "tsx scripts/statskontoret-fetch.ts budget-outturn --source budget-time-series --url <xlsx-url> --persist",
+                    "storage": "analysis/data/statskontoret/budget-time-series/budget-outturn.json"
+                }
+            ]
+        },


+```ts
+import { parseBudgetRows, summarizeBudgetOutturn } from '../scripts/statskontoret-client.js';
+
+const rows = parseBudgetRows(records, { documentType: 'Inkomst' });
+const summary = summarizeBudgetOutturn(rows);
+// summary[0] → { year: 2024, documentType: 'Inkomst', totalOutturn: 700000, totalBudget: 670000, variance: 30000, rowCount: 2 }
+```


pethers · 2026-04-25T10:49:48Z

@copilot apply changes based on the comments in this thread

… formatting Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/29714f50-655f-4c5d-a8c0-9f65516d4ce2 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

github-actions · 2026-04-25T10:54:14Z

🔍 Lighthouse Performance Audit

Category	Score	Status
Performance	85/100	🟡
Accessibility	95/100	🟢
Best Practices	90/100	🟢
SEO	95/100	🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copilot · 2026-04-25T10:55:58Z

@copilot apply changes based on the comments in this thread

Addressed in 1068acb: configurable Statskontoret baseURL now feeds the fetch allowlist guard, network/timeout failures are wrapped as typed StatskontoretError(kind='http') with cause preserved, the JSDoc and use-case import path are corrected, and the Statskontoret inventory JSON is reformatted to 2-space indentation. Added targeted tests; 53/53 Statskontoret tests pass, ESLint/build pass, and parallel validation is clean.

Copilot AI and others added 4 commits April 25, 2026 08:12

Add Statskontoret data integration foundation

a5be249

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/dc62517a-f53c-423f-b327-3d2856b258f8 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

Fix Statskontoret workbook test expectation

9eae74d

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/dc62517a-f53c-423f-b327-3d2856b258f8 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

Address Statskontoret validation feedback

2cd09b1

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/dc62517a-f53c-423f-b327-3d2856b258f8 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

Harden Statskontoret parsing and docs

ec7734f

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/dc62517a-f53c-423f-b327-3d2856b258f8 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

Copilot AI assigned Copilot and pethers Apr 25, 2026

Copilot created this pull request from a session on behalf of pethers April 25, 2026 08:36 View session

github-actions Bot added documentation Documentation updates dependencies Dependency updates testing Test coverage refactor Code refactoring size-xl Extra large change (> 1000 lines) labels Apr 25, 2026

Copilot started work on behalf of pethers April 25, 2026 08:39 View session

Copilot AI requested a review from pethers April 25, 2026 08:42

Copilot started work on behalf of pethers April 25, 2026 08:46 View session

Improve Statskontoret client quality and tests

2bd022a

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/6d0707ac-bee8-4c66-b550-cd699273652c Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

Document Statskontoret integration architecture and threats

64edd9b

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/6d0707ac-bee8-4c66-b550-cd699273652c Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

github-actions Bot added security Security improvements isms ISMS compliance changes iso-27001 ISO 27001 controls nist-csf NIST CSF compliance cis-controls CIS Controls labels Apr 25, 2026

Copilot finished work on behalf of pethers April 25, 2026 08:55

pethers approved these changes Apr 25, 2026

View reviewed changes

Merge branch 'main' into copilot/add-data-support-for-statskontoret

73ec39a

Copilot started work on behalf of pethers April 25, 2026 10:04 View session

Address second review thread on Statskontoret client

b064b08

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/f3c03785-61ec-4ef8-acf8-c12c952c7ded Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

Copilot finished work on behalf of pethers April 25, 2026 10:10

Copilot started work on behalf of pethers April 25, 2026 10:12 View session

Extend Statskontoret integration: budget-outturn parser + CLI command

613146e

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/e7e16b7d-265b-46d9-8628-73a43c059298 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

Fix spelling kalenderár→kalenderar; bump inventory version to 1.1

2b0c849

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/e7e16b7d-265b-46d9-8628-73a43c059298 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

Copilot finished work on behalf of pethers April 25, 2026 10:25

Copilot started work on behalf of pethers April 25, 2026 10:28 View session

Code quality, test coverage (30→51) and docs for statskontoret integr…

6a2f35a

…ation Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/7f40b8bb-7e1e-4b19-9881-f1ad5386d8f0 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

Address code review: explicit :: separator, remove dead sheets object…

f0efab8

…, fix test comment Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/7f40b8bb-7e1e-4b19-9881-f1ad5386d8f0 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

Copilot finished work on behalf of pethers April 25, 2026 10:40

Merge branch 'main' into copilot/add-data-support-for-statskontoret

68c2a81

pethers requested a review from Copilot April 25, 2026 10:41

Copilot started reviewing on behalf of pethers April 25, 2026 10:43 View session

Copilot AI reviewed Apr 25, 2026

View reviewed changes

Copilot started work on behalf of pethers April 25, 2026 10:49 View session

Address review: Statskontoret baseURL guard, typed fetch errors, docs…

1068acb

… formatting Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/29714f50-655f-4c5d-a8c0-9f65516d4ce2 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>

Copilot finished work on behalf of pethers April 25, 2026 10:56

pethers merged commit 7246c68 into main Apr 25, 2026
17 checks passed

pethers deleted the copilot/add-data-support-for-statskontoret branch April 25, 2026 11:07

Conversation

Copilot AI commented Apr 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Apr 25, 2026

🏷️ Automatic Labeling Summary

Label Categories

Uh oh!

github-actions Bot commented Apr 25, 2026

🔍 Lighthouse Performance Audit

Uh oh!

github-actions Bot commented Apr 25, 2026

🔍 Lighthouse Performance Audit

Uh oh!

github-actions Bot commented Apr 25, 2026

🔍 Lighthouse Performance Audit

Uh oh!

github-actions Bot commented Apr 25, 2026

🔍 Lighthouse Performance Audit

Uh oh!

Copilot AI commented Apr 25, 2026

Uh oh!

github-actions Bot commented Apr 25, 2026

🔍 Lighthouse Performance Audit

Uh oh!

github-actions Bot commented Apr 25, 2026

🔍 Lighthouse Performance Audit

Uh oh!

github-actions Bot commented Apr 25, 2026

🔍 Lighthouse Performance Audit

Uh oh!

github-actions Bot commented Apr 25, 2026

🔍 Lighthouse Performance Audit

Uh oh!

github-actions Bot commented Apr 25, 2026

🔍 Lighthouse Performance Audit

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

pethers commented Apr 25, 2026

Uh oh!

github-actions Bot commented Apr 25, 2026

🔍 Lighthouse Performance Audit

Uh oh!

Copilot AI commented Apr 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Apr 25, 2026 •

edited

Loading