Conversation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
Agent-Logs-Url: https://github.com/Hack23/cia/sessions/57d6f1f4-def9-4f9c-b234-5b04ee040d5e Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
📊 JSON Schema Validation ResultsValidation Summary:
Schema Status
📄 Full Report: See uploaded artifacts for detailed validation report ℹ️ Status: Field mismatches are documented and tracked. These represent differences between original projected schemas and actual implemented data. 📄 Data-Validated Schemas: See |
Add implementation status (IMPLEMENTED/COMPUTED/PLANNED/DEPRECATED) for all 142 fields across 5 schemas based on validation run showing 121 mismatches. Key additions: - Status column with icons for every field mapping table - Validation summary showing 21 implemented, 43 computed, 28 planned, 45 deprecated structural grouping fields - Top 25 High-Value Fields prioritization with effort estimates - Mismatch resolution strategy reducing 121 gaps to 18 true gaps - Per-schema unmapped DB columns with priority recommendations - Implementation roadmap (3 sprints) - Field status promotion lifecycle documentation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Add a Field Implementation Status section after the mermaid schema diagram in politician-schema.md. The table categorizes all 55 fields from the class diagram into four statuses: - IMPLEMENTED (11): fields available in current database exports - COMPUTED (13): fields derivable from existing data - PLANNED (11): fields requiring new data sources - STRUCTURAL (20): JSON grouping objects Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
…otations, and enhanced validator Agent-Logs-Url: https://github.com/Hack23/cia/sessions/57d6f1f4-def9-4f9c-b234-5b04ee040d5e Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
📊 JSON Schema Validation ResultsValidation Summary:
Schema Status
📄 Full Report: See uploaded artifacts for detailed validation report ℹ️ Status: Field mismatches are documented and tracked. These represent differences between original projected schemas and actual implemented data. 📄 Data-Validated Schemas: See |
…rrent analysis - Update version to 2.0.0 / 2026-04-05 - Update Overall Assessment table with per-schema coverage percentages - Add Field Status Summary (21 implemented, 50 structural, 71 planned) - Add Mismatch Resolution Strategy (3-category classification) - Add Top 25 Priority Fields for Implementation - Add Remediation Roadmap (3 sprints with effort/impact tracking) - Update Validation Checklist with current status Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
…eld status tests Agent-Logs-Url: https://github.com/Hack23/cia/sessions/57d6f1f4-def9-4f9c-b234-5b04ee040d5e Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
📊 JSON Schema Validation ResultsValidation Summary:
Schema Status
📄 Full Report: See uploaded artifacts for detailed validation report ℹ️ Status: Field mismatches are documented and tracked. These represent differences between original projected schemas and actual implemented data. 📄 Data-Validated Schemas: See |
…d aggregation test Agent-Logs-Url: https://github.com/Hack23/cia/sessions/57d6f1f4-def9-4f9c-b234-5b04ee040d5e Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
📊 JSON Schema Validation ResultsValidation Summary:
Schema Status
📄 Full Report: See uploaded artifacts for detailed validation report ℹ️ Status: Field mismatches are documented and tracked. These represent differences between original projected schemas and actual implemented data. 📄 Data-Validated Schemas: See |
Applied all 3 review comments in
Updated counts: 144 total fields, 24 implemented, 56 structural, 42 computed, 22 planned, 120 mismatches. All 23 tests pass. CodeQL clean. |
| # Pattern matches: +Type fieldName | ||
| # Use composite key (Type:fieldName) to avoid duplicate-name overwrites | ||
| mermaid_pattern = r'\+(\w+)\s+(\w+)' | ||
| for match in re.finditer(mermaid_pattern, content): | ||
| field_type = match.group(1) | ||
| field_name = match.group(2) |
There was a problem hiding this comment.
The mermaid field regex (+(\w+)\s+(\w+)) doesn’t match common type forms like String[] / CommitteeLink[] (the [] breaks the \s+), so array fields in the schema diagrams are silently skipped and never classified/validated. This makes fields_defined/mismatch counts incomplete for several schemas. Update the regex to capture optional [] (and possibly generic forms), and normalize the extracted type so scalar arrays (e.g., String[]) are still treated as scalar while object arrays remain non-scalar.
json-export-specs/FIELD_MAPPING.md
Outdated
| | `decisions` | Container section for decision data | ❌ **STRUCTURAL** | | ||
| | `intelligence` | Container section for intelligence data | ❌ **STRUCTURAL** | | ||
| | `policy` | Container section for policy data | ❌ **STRUCTURAL** | | ||
| | `meetings` | Container section for meeting data (when structural) | ❌ **STRUCTURAL** | |
There was a problem hiding this comment.
meetings is listed as a PLANNED data field above, but also appears under “Structural Fields” as a JSON grouping container. This is contradictory and conflicts with the validator output (where meetings is a planned mismatch field). Remove meetings from the structural list (or rename the structural container if you intended a separate grouping object).
| # Map schema fields to database columns (convert camelCase to snake_case) | ||
| schema_fields = set(schema_info["fields"].keys()) | ||
| # Sort for deterministic output order | ||
| schema_fields = sorted(schema_info["fields"].keys()) | ||
|
|
||
| # Initialize field status tracking | ||
| field_status = { | ||
| "implemented": [], | ||
| "structural": [], | ||
| "computed": [], | ||
| "planned": [] | ||
| } | ||
|
|
||
| # Check if schema fields exist in data (with flexible matching) | ||
| unmapped_schema_fields = [] | ||
| unmapped_data_columns = list(all_columns) | ||
|
|
||
| for field in schema_fields: | ||
| # Try various naming conventions (using set to avoid duplicates) | ||
| possible_names = list({ | ||
| field, | ||
| self._camel_to_snake(field), | ||
| field.lower(), | ||
| field.upper() | ||
| }) | ||
| unmapped_data_columns = sorted(all_columns) |
There was a problem hiding this comment.
You’ve made field-related output deterministic by sorting schema_fields/unmapped_data_columns, but the overall report/JSON still includes lists like matched_views/missing_views whose ordering depends on CSV load/iteration order. For truly stable output (to avoid noisy diffs across runs), consider sorting those view/file lists before writing them into schema_result and validation-results.json.
| - [x] Structural fields (53) documented as JSON grouping objects | ||
| - [x] Computable fields (42) mapped to source DB columns | ||
| - [ ] Planned fields (23) tracked with data source requirements |
There was a problem hiding this comment.
The checklist counts appear out of sync with the validator output in validation-results.json: structural is now 56 (not 53) and planned is 22 (not 23). Update these numbers (and any related text) so the checklist stays consistent with the reported field_status_summary totals.
| - [x] Structural fields (53) documented as JSON grouping objects | |
| - [x] Computable fields (42) mapped to source DB columns | |
| - [ ] Planned fields (23) tracked with data source requirements | |
| - [x] Structural fields (56) documented as JSON grouping objects | |
| - [x] Computable fields (42) mapped to source DB columns | |
| - [ ] Planned fields (22) tracked with data source requirements |
json-export-specs/FIELD_MAPPING.md
Outdated
| `validate_schemas.py` currently reports the raw mismatch total and does **not** yet exclude all non-implemented categories from its counts. | ||
|
|
||
| The following categories are candidates for exclusion from mismatch counts in a future validator enhancement: | ||
| - All ❌ **STRUCTURAL** JSON grouping fields (53 fields) |
There was a problem hiding this comment.
The “Validation Exclusions” bullet still says structural = 53 fields, but the validator summary in validation-results.json (and this PR’s docs elsewhere) reports 56 structural fields. Update this count (and verify planned/computed counts too) to keep the mismatch math consistent.
| - All ❌ **STRUCTURAL** JSON grouping fields (53 fields) | |
| - All ❌ **STRUCTURAL** JSON grouping fields (56 fields) |
|
@copilot apply changes based on the comments in this thread |
…ural list, sort view lists, update all counts - Fix mermaid regex to match Type[] array notation (String[], CommitteeLink[], etc.) - Normalize base type for scalar check (String[] → String is still scalar) - Remove meetings from committee structural fields (it's PLANNED, not structural) - Sort matched_views and missing_views lists for deterministic output - Update SCHEMA_VALIDATION_REPORT.md checklist: structural 56→67, planned 22→35 - Update FIELD_MAPPING.md validation exclusions: structural 56→67 - Update all documentation counts: 168 total fields, 24 implemented, 67 structural, 42 computed, 35 planned, 144 mismatches - Re-run validator to regenerate validation-results.json and auto-generated report - All 23 tests pass Agent-Logs-Url: https://github.com/Hack23/cia/sessions/dafa1381-0a99-4091-bcd6-ad65a11dc1eb Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
📊 JSON Schema Validation ResultsValidation Summary:
Schema Status
📄 Full Report: See uploaded artifacts for detailed validation report ℹ️ Status: Field mismatches are documented and tracked. These represent differences between original projected schemas and actual implemented data. 📄 Data-Validated Schemas: See |
Agent-Logs-Url: https://github.com/Hack23/cia/sessions/dafa1381-0a99-4091-bcd6-ad65a11dc1eb Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
📊 JSON Schema Validation ResultsValidation Summary:
Schema Status
📄 Full Report: See uploaded artifacts for detailed validation report ℹ️ Status: Field mismatches are documented and tracked. These represent differences between original projected schemas and actual implemented data. 📄 Data-Validated Schemas: See |
Applied all 5 review comments in
Updated counts: 168 total fields, 24 implemented, 67 structural, 42 computed, 35 planned, 144 mismatches. All 23 tests pass. CodeQL clean. |
| | **Committees** | | | | | ||
| | `current` | 🔄 PLANNED | Committees | Requires committee membership data | | ||
| | `historical` | 🔄 PLANNED | Committees | Requires historical committee data | | ||
| | `chairPositions` | 🔄 PLANNED | Committees | Requires chair position tracking | | ||
|
|
There was a problem hiding this comment.
The current/historical committee membership fields are shown as 🔄 PLANNED, but in the mermaid diagram they are non-scalar array types (+CommitteeMembership[] current/historical). The validator logic classifies all non-scalar types as STRUCTURAL (and emits composite keys like CommitteeMembership[]:current), so these rows (and the Status Summary counts) should be reconciled to avoid contradicting validation-results.json.
| | 🔄 PLANNED | 8 | Fields requiring new data sources or engine integration | | ||
| | ❌ STRUCTURAL | 22 | JSON grouping objects and non-scalar types, not individual data fields | | ||
|
|
||
| > **Note:** The table above documents all 55 fields from the mermaid diagram (14+11+8+22). The validator tracks 46 of these (14 impl + 19 struct + 11 comp + 2 plan); 9 nested class fields are only documented in this table. See `validation-results.json` for per-field status. |
There was a problem hiding this comment.
The note claims the validator “tracks 46” fields and that “9 nested class fields are only documented in this table”, but the validator’s mermaid regex scans the entire diagram and (per validation-results.json) counts 55 fields for this schema. This note should be updated/removed so it matches the validator behavior and reported counts.
| > **Note:** The table above documents all 55 fields from the mermaid diagram (14+11+8+22). The validator tracks 46 of these (14 impl + 19 struct + 11 comp + 2 plan); 9 nested class fields are only documented in this table. See `validation-results.json` for per-field status. | |
| > **Note:** The table above documents all 55 fields from the mermaid diagram (14+11+8+22). The validator scans the entire mermaid diagram and reports all 55 fields for this schema. See `validation-results.json` for per-field status. |
| | subcategories | 🔄 PLANNED | Labels | Requires new data source | | ||
| | descriptions | ❌ STRUCTURAL | Labels | JSON grouping object | | ||
| | intelligenceTags | 🔄 PLANNED | Labels | Requires new data source | |
There was a problem hiding this comment.
subcategories and intelligenceTags are defined in the mermaid diagram as array types (String[]). The validator treats array fields as structural grouping/containers (and validation-results.json currently lists subcategories/intelligenceTags as STRUCTURAL), but this table marks them as 🔄 PLANNED. Please align these statuses (and the Summary counts) with the validator output/category definitions.
| | subcategories | 🔄 PLANNED | Labels | Requires new data source | | |
| | descriptions | ❌ STRUCTURAL | Labels | JSON grouping object | | |
| | intelligenceTags | 🔄 PLANNED | Labels | Requires new data source | | |
| | subcategories | ❌ STRUCTURAL | Labels | JSON grouping array | | |
| | descriptions | ❌ STRUCTURAL | Labels | JSON grouping object | | |
| | intelligenceTags | ❌ STRUCTURAL | Labels | JSON grouping array | |
| | 🔄 PLANNED | 17 | Requires new data sources | | ||
| | ❌ STRUCTURAL | 18 | JSON grouping objects | | ||
|
|
||
| > **Note:** The table above documents all 51 fields from the mermaid diagram (4+12+17+18). The validator tracks 43 of these (4 impl + 18 struct + 12 comp + 9 plan); 8 nested class fields are only documented in this table. See `validation-results.json` for per-field status. |
There was a problem hiding this comment.
The note says the validator “tracks 43” fields and that 8 nested fields are only documented here, but validation-results.json reports fields_defined: 51 for the Party schema, which implies the validator is already counting all mermaid fields. Please update/remove this note to avoid contradicting the validator output.
| > **Note:** The table above documents all 51 fields from the mermaid diagram (4+12+17+18). The validator tracks 43 of these (4 impl + 18 struct + 12 comp + 9 plan); 8 nested class fields are only documented in this table. See `validation-results.json` for per-field status. | |
| > **Note:** The table above documents all 51 fields from the mermaid diagram (4+12+17+18), which matches the Party schema `fields_defined: 51` reported in `validation-results.json`. See that file for per-field status details. |
| **Date:** 2026-04-05 | ||
| **Status:** Analysis Complete - Remediation Plan Provided | ||
| **Validated Against:** service.data.impl/src/main/resources/full_schema.sql | ||
| **Total Schema Mismatches:** 120 (across 4 entity schemas) |
There was a problem hiding this comment.
This report header states Total Schema Mismatches: 120, but the updated validator output and the Field Status Summary in the same document indicate 144 mismatches (168 total fields − 24 implemented). Please reconcile the 120 vs 144 mismatch totals so readers don’t get conflicting metrics.
| **Total Schema Mismatches:** 120 (across 4 entity schemas) | |
| **Total Schema Mismatches:** 144 (across 4 entity schemas) |
|
@copilot apply changes based on the comments in this thread |
… output - politician: current/historical → STRUCTURAL (CommitteeMembership[] non-scalar), summary 8→6 planned, 22→24 structural - politician: note updated to reflect validator tracks all 55 fields (not 46) - party: subcategories/intelligenceTags → STRUCTURAL (String[] array types per validator) - party: history/regions → STRUCTURAL (ElectionHistory[]/RegionalSupport[] non-scalar) - party: summary 17→13 planned, 18→22 structural; note updated to match fields_defined: 51 - SCHEMA_VALIDATION_REPORT.md: Total Schema Mismatches 120 → 144 (168 - 24 implemented) Agent-Logs-Url: https://github.com/Hack23/cia/sessions/87ca677d-6eed-4545-9caa-615aaee059c1 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
📊 JSON Schema Validation ResultsValidation Summary:
Schema Status
📄 Full Report: See uploaded artifacts for detailed validation report ℹ️ Status: Field mismatches are documented and tracked. These represent differences between original projected schemas and actual implemented data. 📄 Data-Validated Schemas: See |
Applied all 5 review comments in
All 23 tests pass. |
|
Description
The 5 JSON export schemas define 168 fields but only 24 exist in database sample data (144 mismatches). Analysis reveals most "mismatches" are either JSON structural grouping objects (67) or fields derivable from existing DB columns (42) — not true gaps. This PR classifies every field into 4 validator-backed categories (implemented/structural/computed/planned), annotates all schema docs, enhances the validator to report status with fully deterministic output, fixes a critical view-mapping bug, a duplicate field-name handling bug, and an array type parsing bug in the validator, and provides a prioritized implementation roadmap.
A critical bug was discovered and fixed in
validate_schemas.py: the ministryschema_to_viewsmapping usedriksdagen_government(English spelling) but the actual CSV files and DB views useriksdagen_goverment(matching the database identifiers). This caused the validator to miss 6 ministry CSV data files entirely. Additional missing view patterns were added for committee (committee_productivity) and ministry (ministry_productivity_matrix,ministry_risk_evolution), increasing implemented fields from 21 to 24.A second bug was fixed in the mermaid field parser: duplicate field names across different classes/paths (e.g., politician has both
+String partyand+PartyLink party) would silently overwrite each other. Non-scalar types (PartyLink, Trends, VoteAlignment, etc.) are now tracked using composite keys (e.g.,PartyLink:party) and are always classified as STRUCTURAL. ASCALAR_TYPESconstant distinguishes scalar field types from non-scalar/link types. ThePartyLink partyentry was removed frompolitician-schema-validated.mdsince CSV sample data validates scalar columns, not nested link objects.A third bug was fixed in the mermaid field regex: the pattern
+(\w+)\s+(\w+)did not match array type forms likeString[]orCommitteeLink[](the[]broke the\w+match), so array fields in schema diagrams were silently skipped and never classified/validated. The regex now captures optional[]notation, and the extracted base type is normalized (e.g.,String[]→Stringis still scalar,CommitteeLink[]remains non-scalar/STRUCTURAL).Additionally,
meetingswas removed from the committee structural fields list in FIELD_MAPPING.md — it was contradictorily listed as both PLANNED and STRUCTURAL, and the validator classifies it as planned. All view/file lists (matched_views,missing_views) are now sorted before writing tovalidation-results.jsonfor fully deterministic output across runs.Field classifications distinguish between nested object types (e.g.,
+Trends trend,+VoteAlignment alignment,+Productivity productivity) which are STRUCTURAL, and leaf metrics (e.g.,+Integer committeeChairs,+Float strengthScore) which are COMPUTED or PLANNED. All doc table classifications — including validated schema files, FIELD_MAPPING.md per-schema sections, and summary reports — are fully reconciled withvalidation-results.jsonoutput across 9 rounds of review alignment. All structural JSON grouping fields consistently use ❌ STRUCTURAL labeling (not DEPRECATED) across all documentation sections. Ministry view references use the actual DB view identifiers (view_riksdagen_goverment*) throughout all sections including the Top 25 implementation roadmap table.Type of Change
Primary Changes
Political Analysis
Technical Changes
Impact Analysis
Political Analysis Impact
Technical Impact
Testing
23 tests pass (5 new/updated):
test_structural_fields_defined— verifies STRUCTURAL_FIELDS constant (includingtrend,alignment)test_computed_fields_defined— verifies COMPUTED_FIELDS constant (includingid,committeeChairs,strengthScore)test_field_status_in_validation_results— verifies results structure with all 4 categoriestest_field_classification_with_data— end-to-end classification with aggregation across structural/computed/plannedtest_mismatch_includes_status— verifies STRUCTURAL/COMPUTED/PLANNED status in mismatch entriesDocumentation
Key changes:
validate_schemas.py— Fixed critical ministry view-mapping bug (riksdagen_government→riksdagen_govermentmatching actual DB spelling). Fixed duplicate field-name handling: non-scalar types (PartyLink, Trends, VoteAlignment, etc.) now use composite keys (Type:name) to prevent overwriting same-named scalar fields. Fixed mermaid regex to matchType[]array notation; base type is normalized for scalar check (String[]→Stringis scalar,CommitteeLink[]remains non-scalar). AddedSCALAR_TYPESconstant to distinguish scalar vs non-scalar field types; non-scalar types are always classified as STRUCTURAL. Added missing view patterns:committee_productivity,ministry_productivity_matrix,ministry_risk_evolution. Enhanced with 3-way mismatch classification, fully deterministic output (sortedschema_fields,unmapped_data_columns,matched_views,missing_views), and dynamic file counts:FIELD_MAPPING.md— Every field now has ✅ IMPLEMENTED / 🔀 COMPUTED / 🔄 PLANNED / ❌ STRUCTURAL status. Status legend maps directly to validator output categories. Validation Summary table aligned with validator output (67/42/35). Per-schema sections reconciled withvalidation-results.json: politician implemented fields includeriskScore,riskLevel,attendanceRate; party implemented includestotalVotes; committee implemented includesregularMembers,totalMembers; ministry implemented includesname,id. All structural/computed/planned lists match validator per-schema breakdowns.meetingsremoved from committee structural list (it's PLANNED per validator). Committee and ministry structural field sections consistently use ❌ STRUCTURAL labeling. Ministry view references corrected to actual DB identifiers (view_riksdagen_goverment*) across all sections including the Top 25 implementation roadmap table. Mismatch reduction strategy table uses 67 structural fields with cascading count adjustments. Validation exclusions count updated to 67 structural fields.4 schema files (
politician-,party-,committee-,ministry-schema.md) — Added📋 Field Implementation Statussection after each mermaid diagram. Nested object types correctly classified as STRUCTURAL. Leaf metrics classified as COMPUTED. Non-scalar link types (e.g.,MinistryLink:ministry) reclassified as STRUCTURAL. Array types (e.g.,CommitteeMembership[] current/historical,ElectionHistory[] history,RegionalSupport[] regions,CommitteeLink[] committees) classified as STRUCTURAL per validator logic. Politician schema:current/historicalreclassified from PLANNED to STRUCTURAL (non-scalarCommitteeMembership[]arrays); field count note updated to match validator's 55fields_defined. Party schema:subcategories/intelligenceTagsreclassified from PLANNED to STRUCTURAL (array types per validator);history/regionsreclassified from PLANNED to STRUCTURAL (non-scalarElectionHistory[]/RegionalSupport[]arrays); field count note updated to match validator's 51fields_defined. Summary counts in both schemas reconciled with per-schema breakdowns invalidation-results.json. Committee schema:regularMembers/totalMembersnow IMPLEMENTED (found inview_committee_productivity). Ministry schema:idnow IMPLEMENTED (found inview_riksdagen_goverment_proposals).4 validated schemas (
*-validated.md) — Field counts aligned withvalidation-results.jsonper-schema breakdowns.PartyLink partyremoved frompolitician-schema-validated.md(CSV validates scalars only; scalarString partyretained).party-schema-validated.mdmermaid diagram updated to includetotalVotes. Duplicate## Migration Notesheadings removed.SCHEMA_VALIDATION_REPORT.md— Uses dynamicfiles_analyzedcount. Total Schema Mismatches corrected to 144 (168 − 24 implemented). Field Status Summary shows full 4-category breakdown with corrected numbers: 24 implemented, 67 structural, 42 computed, 35 planned. Checklist counts updated to match validator output. Mismatch resolution strategy section updated.VALIDATED_SCHEMAS_SUMMARY.md— Numbers aligned with actual validator output (24 implemented, 67 structural, 42 computed, 35 planned)..gitignore— Added__pycache__/,*.pyc, and.pytest_cache/to prevent committing Python build artifacts.Screenshots
N/A
Related Issues
Related to #893
Checklist
Additional Notes
Mismatch breakdown from validator output: 67 structural (JSON grouping objects like
attributes,labels,trend,alignment,productivity,budget,predictions, non-scalar link types likePartyLink:party,MinistryLink:ministry, and array link types likeCommitteeLink[]:committees,CommitteeMembership[]:current,ElectionHistory[]:history,RegionalSupport[]:regions) + 42 computed (derivable fields likefullName,totalMembers,committeeChairs,strengthScore,established) + 35 planned (fields needing new data sources) = 144 total mismatches (24 fields implemented, 168 total fields). The critical view-mapping fix corrected the ministryschema_to_viewsmapping fromriksdagen_governmenttoriksdagen_goverment(matching the actual DB view spelling infull_schema.sql), which unlocked 6 additional ministry CSV data files. The composite-key fix prevents non-scalar types from silently overwriting same-named scalar fields — non-scalar types are tracked asType:namekeys and always classified as STRUCTURAL. The array regex fix ensuresType[]fields (e.g.,String[],CommitteeLink[]) are parsed and classified rather than silently skipped. Committee gained 2 implemented fields (regularMembers,totalMembersfromview_committee_productivity) and ministry gained 1 (idfromview_riksdagen_goverment_proposals). All documentation tables across validated schemas, FIELD_MAPPING.md per-schema sections, schema field implementation status tables, and summary reports reflect these validator-verified numbers. Per-schema field implementation status tables in politician-schema.md and party-schema.md reconciled: non-scalar array types (CommitteeMembership[],ElectionHistory[],RegionalSupport[]) correctly classified as STRUCTURAL; scalar array types (String[]forsubcategories/intelligenceTags) also classified as STRUCTURAL per validator output; field count notes updated to match validatorfields_definedtotals (55 for politician, 51 for party). All view/file lists are sorted for fully deterministic output across runs. All structural JSON grouping fields consistently use ❌ STRUCTURAL labeling across all documentation sections. Ministry source view references use the actual DB identifiers (view_riksdagen_goverment,view_riksdagen_goverment_role_member) throughout all documentation sections including the Top 25 implementation roadmap table.Security Considerations
Release Notes
Fixed critical validator view-mapping bug (ministry
riksdagen_government→riksdagen_govermentmatching actual DB spelling), fixed duplicate field-name handling (non-scalar types now use compositeType:namekeys and are classified as STRUCTURAL), fixed mermaid regex to parseType[]array fields (previously silently skipped), removed contradictorymeetingsfrom committee structural list (it's PLANNED per validator), sorted all view/file lists for fully deterministic output, and added missing view patterns (committee_productivity,ministry_productivity_matrix,ministry_risk_evolution). RemovedPartyLink partyfrom politician validated schema (CSV validates scalars only). Reconciled all per-schema field implementation status tables with validator output — non-scalar array types (CommitteeMembership[],ElectionHistory[],RegionalSupport[]) and scalar array types (String[]) correctly classified as STRUCTURAL; field count notes aligned with validatorfields_definedtotals; total mismatches header corrected to 144. Updated validation results: 144 mismatches (168 total fields), 24 implemented, 67 structural, 42 computed, 35 planned. Classified all JSON export schema field mismatches into 4 validator-backed categories, added field status annotations to all schema and validated schema documentation, fully reconciled all per-schema field lists and counts withvalidation-results.jsonoutput across 9 rounds of review alignment, ensured consistent STRUCTURAL labeling across all documentation sections, added.gitignorefor Python artifacts, and created a prioritized 25-field implementation roadmap.