Conversation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
…ed) verified against full_schema.sql Agent-Logs-Url: https://github.com/Hack23/cia/sessions/a912fc77-6eb0-4480-bf8a-ac8d33d14792 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
…T_DOCUMENT.md Agent-Logs-Url: https://github.com/Hack23/cia/sessions/a912fc77-6eb0-4480-bf8a-ac8d33d14792 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Documentation reconciliation to align all view-count references with full_schema.sql ground truth (110 total views: 77 regular + 33 materialized), ensuring consistent cross-document navigation and metrics for the CIA platform’s intelligence layer.
Changes:
- Updated view totals and category distributions across core intelligence documentation.
- Expanded/adjusted view category breakdowns (incl. election cycle and seasonal/election-year categories) in the catalog and data-flow mapping.
- Refreshed cross-referenced docs that previously cited stale totals.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Updates high-level README references to the correct total view count. |
| INTELLIGENCE_DATA_FLOW.md | Updates view totals and category table to reflect the audited inventory. |
| DATABASE_VIEW_INTELLIGENCE_CATALOG.md | Updates executive summary, key stats, inventory/category counts, and validation history to 110 views. |
| DATA_QUALITY_MONITORING_DASHBOARD.md | Updates dashboard metrics and architecture diagrams to the corrected view totals. |
| DATA_ANALYSIS_INTOP_OSINT.md | Updates analysis doc references to the corrected total view count. |
| CHANGELOG_INTELLIGENCE.md | Updates changelog references to the correct view inventory size. |
| BUSINESS_PRODUCT_DOCUMENT.md | Updates business documentation references to the corrected distinct view total. |
Comments suppressed due to low confidence (2)
INTELLIGENCE_DATA_FLOW.md:101
- The data-flow diagram header was updated to "Database Views - 110 Total", but the category nodes inside the same subgraph still show the old per-category counts (e.g., Politician 8, Party 12, Vote Data 20, etc.). Please update the node counts (and consider adding the new Election Cycle / Seasonal categories if they’re part of the 110) so the diagram matches the revised catalog numbers.
subgraph DATABASE["🗄️ Database Views - 110 Total"]
direction LR
V1["<b>Politician Views</b><br/>8 views<br/>Performance tracking"]
V2["<b>Party Views</b><br/>12 views<br/>Party analytics"]
V3["<b>Committee Views</b><br/>12 views<br/>Legislative work"]
V4["<b>Ministry Views</b><br/>7 views<br/>Government oversight"]
V5["<b>Vote Data Views</b><br/>20 views<br/>Voting patterns"]
V6["<b>Document Views</b><br/>7 views<br/>Legislative output"]
V7["<b>Intelligence Views</b><br/>6 views<br/>Risk & anomaly"]
BUSINESS_PRODUCT_DOCUMENT.md:154
- This section updates the distinct view total to 110, but the validation line still says "✅ Validated 2026-01-19 (v1.61)". Please update the validation date/version to match the new 2026-04-05 audit (v1.80) so consumers don’t assume the category counts were validated under the older schema.
**Total Database Views:** 110 (77 regular + 33 materialized)
_Note: Category view counts above are not mutually exclusive; a single database view may support multiple intelligence frameworks and can therefore appear in more than one category. The total of 110 refers to distinct database views._
**Framework Coverage:** All 6 intelligence frameworks comprehensively supported
**Documentation:** 100% coverage in [DATABASE_VIEW_INTELLIGENCE_CATALOG.md](DATABASE_VIEW_INTELLIGENCE_CATALOG.md)
**Validation Status:** ✅ Validated 2026-01-19 (v1.61)
| **Note**: Total view count remains 96. Four views (party_summary, party_longitudinal_performance, party_coalition_evolution, party_electoral_trends) were recreated in v1.61 after being dropped in v1.53/v1.6 but never recreated, causing JPA entity mismatch. | ||
| **Note**: Total view count updated from 96 to 110 after full audit against full_schema.sql (v1.80). Added 5 new-style materialized views (mv_*), 4 career/trajectory politician views, 7 seasonal/election year views, and 1 decision temporal trends view that were previously undercounted. Four views (party_summary, party_longitudinal_performance, party_coalition_evolution, party_electoral_trends) were recreated in v1.61 after being dropped in v1.53/v1.6 but never recreated, causing JPA entity mismatch. | ||
|
|
||
| ### Key Statistics (UPDATED 2026-01-19) |
There was a problem hiding this comment.
The heading still says "Key Statistics (UPDATED 2026-01-19)" even though the surrounding summary states the full audit/validation happened 2026-04-05 (v1.80). Updating this heading (and thus its generated anchor) will avoid confusing timestamps and broken deep-links from other docs.
| ### Key Statistics (UPDATED 2026-01-19) | |
| ### Key Statistics (UPDATED 2026-04-05) |
| | **Views Documented (Detailed)** | 15 | ✅ Views with comprehensive examples and performance characteristics | | ||
| | **Views Documented (Structured)** | 95 | Purpose, metrics, queries, product mappings | | ||
| | **Documentation Coverage** | 100% | All 110 views documented | | ||
| | **Intelligence Views** | 7 | Advanced analytical views (risk, anomaly, influence, crisis, momentum, dashboard, temporal trends) | |
There was a problem hiding this comment.
Key Statistics lists Intelligence Views as 7, but the inventory section shows "Intelligence & Risk Views (8 views)" (and includes two temporal-trends views). Please reconcile this count so category totals are consistent within the catalog.
| | **Intelligence Views** | 7 | Advanced analytical views (risk, anomaly, influence, crisis, momentum, dashboard, temporal trends) | | |
| | **Intelligence Views** | 8 | Advanced analytical views (risk, anomaly, influence, crisis, momentum, dashboard, temporal trends) | |
| **Detailed Documentation (📖):** 15 | ||
| **Structured Documentation (📝):** 95 | ||
| **Documentation Coverage:** 100% | ||
| **Materialized Views:** 28 |
There was a problem hiding this comment.
The inventory summary still states "Materialized Views: 28" even though the catalog now claims 33 materialized views (28 legacy + 5 mv_*). This should be updated to avoid conflicting totals on the same page.
| **Materialized Views:** 28 | |
| **Materialized Views:** 33 |
| The documentation achieved 100% coverage through systematic validation and remediation: | ||
|
|
||
| - **2025-11-20**: Initial validation revealed 10.98% coverage (9/82 views) | ||
| - **2025-11-20**: Initial validation revealed 10.98% coverage (9/110 views) |
There was a problem hiding this comment.
In Coverage Progression, the initial validation is described as "10.98% coverage (9/110 views)", but the Historical Validations table directly above states there were 82 total views on 2025-11-20. The denominator should match the point-in-time total (likely 9/82) to keep the history internally consistent.
| - **2025-11-20**: Initial validation revealed 10.98% coverage (9/110 views) | |
| - **2025-11-20**: Initial validation revealed 10.98% coverage (9/82 views) |
| **Last Updated:** 2025-12-10 | ||
| **Last Validated Against Schema:** 2025-11-25 | ||
| **Last Validated Against Schema:** 2026-04-05 | ||
| **Next Review:** 2025-12-25 (monthly review recommended) |
There was a problem hiding this comment.
Document Metadata still shows Version 2.2 / Date 2025-12-10 / Last Updated 2025-12-10, but this PR adds a 3.0 change-log entry dated 2026-04-05 and updates counts to v1.80. Please update metadata fields (version/date/last-updated/next-review) so the document header reflects the current revision.
DATA_ANALYSIS_INTOP_OSINT.md
Outdated
| - **Regular Views**: 54 views (100% documented) | ||
| - **Materialized Views**: 28 views (100% documented) |
There was a problem hiding this comment.
This section updates the total to 110 views, but the breakdown directly below still states 54 regular + 28 materialized. Please update these sub-counts to match the new totals (77 regular + 33 materialized) to avoid contradictory metrics in the same summary.
| - **Regular Views**: 54 views (100% documented) | |
| - **Materialized Views**: 28 views (100% documented) | |
| - **Regular Views**: 77 views (100% documented) | |
| - **Materialized Views**: 33 views (100% documented) |
INTELLIGENCE_DATA_FLOW.md
Outdated
| | **Intelligence Views** | 8 | Pattern, Predictive, All | ⭐⭐⭐⭐⭐ VERY HIGH | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md#intelligence--risk-views) | | ||
| | **Election Cycle Views** | 6 | Temporal, Predictive, Network | ⭐⭐⭐⭐⭐ VERY HIGH | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md#election-cycle-views) | | ||
| | **Seasonal/Election Year Views** | 7 | Temporal, Pattern | ⭐⭐⭐⭐ HIGH | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md#seasonal--election-year-views) | | ||
| | **Application Views** | 14 | Temporal, Pattern | ⭐⭐ LOW | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md#application--audit-views) | |
There was a problem hiding this comment.
Several documentation links in the category summary table appear to use anchors that don’t exist in DATABASE_VIEW_INTELLIGENCE_CATALOG.md (e.g., #intelligence--risk-views, #election-cycle-views, #seasonal--election-year-views). Please update these to the actual section anchors in the catalog (or add stable headings/anchors in the catalog) so the links resolve correctly.
| | **Intelligence Views** | 8 | Pattern, Predictive, All | ⭐⭐⭐⭐⭐ VERY HIGH | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md#intelligence--risk-views) | | |
| | **Election Cycle Views** | 6 | Temporal, Predictive, Network | ⭐⭐⭐⭐⭐ VERY HIGH | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md#election-cycle-views) | | |
| | **Seasonal/Election Year Views** | 7 | Temporal, Pattern | ⭐⭐⭐⭐ HIGH | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md#seasonal--election-year-views) | | |
| | **Application Views** | 14 | Temporal, Pattern | ⭐⭐ LOW | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md#application--audit-views) | | |
| | **Intelligence Views** | 8 | Pattern, Predictive, All | ⭐⭐⭐⭐⭐ VERY HIGH | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md) | | |
| | **Election Cycle Views** | 6 | Temporal, Predictive, Network | ⭐⭐⭐⭐⭐ VERY HIGH | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md) | | |
| | **Seasonal/Election Year Views** | 7 | Temporal, Pattern | ⭐⭐⭐⭐ HIGH | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md) | | |
| | **Application Views** | 14 | Temporal, Pattern | ⭐⭐ LOW | [View Docs](DATABASE_VIEW_INTELLIGENCE_CATALOG.md) | |
DATA_QUALITY_MONITORING_DASHBOARD.md
Outdated
| | **[OSINT Data Analysis](DATA_ANALYSIS_INTOP_OSINT.md)** | 🎯 OSINT Quality | Source data quality metrics, 6 analytical frameworks | [View](DATA_ANALYSIS_INTOP_OSINT.md#core-platform-metrics-verified-2025-11-28) | | ||
| | **[Schema Maintenance Guide](service.data.impl/README-SCHEMA-MAINTENANCE.md)** | 🔧 Health Checks | Database health check procedures, validation scripts | [View](service.data.impl/README-SCHEMA-MAINTENANCE.md#health-check-and-validation) | | ||
| | **[View Intelligence Catalog](DATABASE_VIEW_INTELLIGENCE_CATALOG.md)** | 🗄️ View Validation | 84 database views catalog with validation status | [View](DATABASE_VIEW_INTELLIGENCE_CATALOG.md#key-statistics-reverified-2025-11-25) | | ||
| | **[View Intelligence Catalog](DATABASE_VIEW_INTELLIGENCE_CATALOG.md)** | 🗄️ View Validation | 110 database views catalog with validation status | [View](DATABASE_VIEW_INTELLIGENCE_CATALOG.md#key-statistics-updated-v180) | |
There was a problem hiding this comment.
This link points to DATABASE_VIEW_INTELLIGENCE_CATALOG.md#key-statistics-updated-v180, but the catalog’s Key Statistics heading currently generates a different anchor (and there is no explicit key-statistics-updated-v180 anchor). Please update the link target (or add a stable anchor in the catalog) to avoid a broken cross-reference.
| | **[View Intelligence Catalog](DATABASE_VIEW_INTELLIGENCE_CATALOG.md)** | 🗄️ View Validation | 110 database views catalog with validation status | [View](DATABASE_VIEW_INTELLIGENCE_CATALOG.md#key-statistics-updated-v180) | | |
| | **[View Intelligence Catalog](DATABASE_VIEW_INTELLIGENCE_CATALOG.md)** | 🗄️ View Validation | 110 database views catalog with validation status | [View](DATABASE_VIEW_INTELLIGENCE_CATALOG.md) | |
DATA_QUALITY_MONITORING_DASHBOARD.md
Outdated
| │ - 93 tables, 84 views (56 regular + 28 mat views) │ | ||
| │ - 93 tables, 110 views (77 regular + 33 mat views) │ | ||
| │ - 20 GB, 5.6M rows │ | ||
| │ - Schema v1.37 with health monitoring │ |
There was a problem hiding this comment.
The diagram now states 110 views, but still labels the schema as "v1.37". Elsewhere in this PR the view audit is tied to v1.80; please reconcile the schema version shown here (either update to the current schema version or clarify that the diagram reflects an older snapshot).
| │ - Schema v1.37 with health monitoring │ | |
| │ - Schema v1.80 with health monitoring │ |
|
@copilot apply changes based on the comments in this thread |
…ncile Intelligence Views count (7→8), fix materialized count (28→33), correct point-in-time denominator (9/82), update document metadata to v3.0, fix DATA_ANALYSIS sub-counts (54→77, 28→33), remove broken anchors in cross-doc links, update schema version (v1.37→v1.80) Agent-Logs-Url: https://github.com/Hack23/cia/sessions/2d87c40f-c630-4aa4-97ce-c9c715b9eebe Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Applied all 9 review comments in commit
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.
Comments suppressed due to low confidence (1)
INTELLIGENCE_DATA_FLOW.md:103
- In the Mermaid diagram, the per-category view counts (V1–V8) still show the old numbers (e.g., Politician 8, Party 12, Ministry 7, Vote 20, etc.) and don’t include the new Election Cycle / Seasonal categories. This makes the diagram inconsistent with the updated 110-total and the summary table below; update the node counts (and add nodes if needed) to match the new category totals.
subgraph DATABASE["🗄️ Database Views - 110 Total"]
direction LR
V1["<b>Politician Views</b><br/>8 views<br/>Performance tracking"]
V2["<b>Party Views</b><br/>12 views<br/>Party analytics"]
V3["<b>Committee Views</b><br/>12 views<br/>Legislative work"]
V4["<b>Ministry Views</b><br/>7 views<br/>Government oversight"]
V5["<b>Vote Data Views</b><br/>20 views<br/>Voting patterns"]
V6["<b>Document Views</b><br/>7 views<br/>Legislative output"]
V7["<b>Intelligence Views</b><br/>6 views<br/>Risk & anomaly"]
V8["<b>Application Views</b><br/>10 views<br/>Platform usage"]
end
| **Last Updated:** 2026-04-05 (Full audit: corrected view count from 96 to 110 — 77 regular + 33 materialized — verified against full_schema.sql) | ||
| **Last Validated**: 2026-04-05 (v1.80 with 110 views) | ||
| **Validation Method**: Automated schema validation via validate-view-documentation.sh | ||
| **Schema Source**: service.data.impl/src/main/resources/full_schema.sql + db-changelog-1.61.xml | ||
| **Documentation Coverage**: 100% (96/96 views) | ||
| **Schema Source**: service.data.impl/src/main/resources/full_schema.sql + db-changelog-1.80.xml | ||
| **Documentation Coverage**: 100% (110/110 views) |
There was a problem hiding this comment.
The file’s metadata is internally inconsistent: this section states Last Updated/Validated 2026-04-05 (v1.80), but the document header at the top still shows an older Last Updated date/version. Please reconcile the top-of-file metadata block with the new document metadata/versioning used elsewhere in this update.
| | **Vote Summary Views** | 15 | Daily, weekly, monthly, annual ballot summaries (ballot, party, politician) | | ||
| | **Application Event Views** | 12 | User behavior tracking (daily, weekly, monthly, annual) | | ||
| | **Document Views** | 7 | Politician and party document productivity | |
There was a problem hiding this comment.
Key Statistics counts don’t match the inventory below: the inventory lists Document Views as 9 and Vote Data Views as 16, but this table still shows Document Views = 7 and only references Vote Summary Views = 15. Please update/rename these metrics so the counts align with the documented category totals.
| | **Vote Summary Views** | 15 | Daily, weekly, monthly, annual ballot summaries (ballot, party, politician) | | |
| | **Application Event Views** | 12 | User behavior tracking (daily, weekly, monthly, annual) | | |
| | **Document Views** | 7 | Politician and party document productivity | | |
| | **Vote Data Views** | 16 | Daily, weekly, monthly, annual ballot and voting data summaries (ballot, party, politician) | | |
| | **Application Event Views** | 12 | User behavior tracking (daily, weekly, monthly, annual) | | |
| | **Document Views** | 9 | Politician and party document productivity | |
| ### Current Status | ||
|
|
||
| **Last Validated**: 2025-11-25 | ||
| **Validation Method**: Automated schema validation via validate-view-documentation.sh | ||
| **Schema Source**: service.data.impl/src/main/resources/full_schema.sql | ||
| **Coverage**: 100% (84/84 views documented) | ||
| **Coverage**: 100% (110/110 views documented) | ||
| **Health Score**: 82/100 (per schema-health-check.sql) |
There was a problem hiding this comment.
The “Current Status” block still says “Last Validated: 2025-11-25”, which contradicts the earlier 2026-04-05 validation/date updates and the added 2026-04-05 entry in the validation history. Update this block so it reflects the current audit/validation date.
BUSINESS_PRODUCT_DOCUMENT.md
Outdated
| @@ -126,7 +126,7 @@ This section provides direct links to JSON specifications defining the data stru | |||
| | **Election Cycle Analysis (v1.59-v1.60)** | [intelligence-schema.md](json-export-specs/schemas/intelligence-schema.md) | `view_riksdagen_election_proximity_trends`, `view_riksdagen_election_year_behavioral_patterns`, `view_riksdagen_pre_election_quarterly_activity` | Weekly | 9 election views | | |||
There was a problem hiding this comment.
The “Election Cycle Analysis (v1.59-v1.60)” row lists 3 primary views but claims “9 election views”. This count appears inconsistent with the updated catalog’s election/seasonal view inventory; adjust the number (or clarify what it’s counting) to avoid misleading totals.
| | **Election Cycle Analysis (v1.59-v1.60)** | [intelligence-schema.md](json-export-specs/schemas/intelligence-schema.md) | `view_riksdagen_election_proximity_trends`, `view_riksdagen_election_year_behavioral_patterns`, `view_riksdagen_pre_election_quarterly_activity` | Weekly | 9 election views | | |
| | **Election Cycle Analysis (v1.59-v1.60)** | [intelligence-schema.md](json-export-specs/schemas/intelligence-schema.md) | `view_riksdagen_election_proximity_trends`, `view_riksdagen_election_year_behavioral_patterns`, `view_riksdagen_pre_election_quarterly_activity` | Weekly | 3 election views | |
DATA_ANALYSIS_INTOP_OSINT.md
Outdated
| **View Dependencies**: | ||
| - 84 database views available (56 regular + 28 materialized) | ||
| - 110 database views available (77 regular + 33 materialized) | ||
| - 7 intelligence-specific views (risk, anomaly, influence, crisis, momentum, dashboard, temporal trends) |
There was a problem hiding this comment.
This section now states 110 total views, but still claims “7 intelligence-specific views …”. The updated view catalog lists 8 “Intelligence & Risk Views”; please reconcile this count (and/or the list description) so it matches the authoritative catalog.
| - 7 intelligence-specific views (risk, anomaly, influence, crisis, momentum, dashboard, temporal trends) | |
| - 8 intelligence & risk views (see `DATABASE_VIEW_INTELLIGENCE_CATALOG.md` for the authoritative catalog and validation history) |
| #### Database View Categories (110 Total Views) | ||
|
|
||
| **View Distribution by Intelligence Framework:** | ||
|
|
There was a problem hiding this comment.
Within this section, the heading/table was updated to 110 total views, but earlier in the same “Product-to-Data Mapping” area the “Complete View Reference” sentence still claims 112 views. Please update that earlier statement to 110 so the document doesn’t contradict itself.
|
@copilot apply changes based on the comments in this thread |
…pdate Current Status date, fix election view count (9→3), fix intelligence views (7→8), reconcile all stale 112→110 references in BUSINESS_PRODUCT_DOCUMENT.md Agent-Logs-Url: https://github.com/Hack23/cia/sessions/7aca9138-6dfa-4389-a7d2-04459fe95979 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Applied all 6 round-2 review comments in commit
|
|
Description
Full audit of database views against
full_schema.sqlground truth revealed the catalog claimed 96 views (68 regular + 28 materialized) but the actual schema contains 110 views (77 regular + 33 materialized). The catalog already had 110 individual view sections—only the summary counts, category tables, and cross-referenced docs were stale.Key discrepancies:
mv_*materialized views (v1.76) never countedmv_ministry_decision_impactandmv_party_decision_flowuncountedFiles updated (7):
DATABASE_VIEW_INTELLIGENCE_CATALOG.md— counts, category tables (+23 missing views), new Election Cycle and Seasonal/Election Year categories, document metadata bumped to version 3.0 (2026-04-05), version history 3.0, Current Status validation date updatedREADME.md,INTELLIGENCE_DATA_FLOW.md,CHANGELOG_INTELLIGENCE.md,BUSINESS_PRODUCT_DOCUMENT.md,DATA_ANALYSIS_INTOP_OSINT.md,DATA_QUALITY_MONITORING_DASHBOARD.md— stale count references, broken anchor links, and schema version correctedReview feedback addressed (15 items across 2 rounds):
Round 1 (9 items):
Round 2 (6 items):
Type of Change
Primary Changes
Political Analysis
Technical Changes
Impact Analysis
Political Analysis Impact
Technical Impact
Testing
Documentation-only change; no code or runtime behavior affected.
Documentation
Screenshots
N/A
Related Issues
Checklist
Additional Notes
Historical changelog entries (version history in the catalog, CHANGELOG_INTELLIGENCE.md release notes) were intentionally left with their original counts since they represent point-in-time snapshots. Cross-document anchor links were simplified to point to the catalog root to avoid breakage from heading changes. All stale "112 views" references in BUSINESS_PRODUCT_DOCUMENT.md (from a prior documentation pass) were corrected to 110 across Mermaid diagrams, tables, and prose.
Security Considerations
Release Notes
Reconciled database view documentation: corrected total view count from 96 to 110 (77 regular + 33 materialized) after full audit against
full_schema.sql. Added 23 previously uncounted views to category tables including new Election Cycle (6) and Seasonal/Election Year (7) categories. Updated document metadata to version 3.0, fixed broken cross-document anchor links, and reconciled all stale 112→110 references in BUSINESS_PRODUCT_DOCUMENT.md.