Skip to content

feat(motions): implement search_dokument_fulltext, analyze_g0v_by_department, and search_anforanden#603

Merged
pethers merged 4 commits intomainfrom
copilot/add-full-text-analysis-tools
Feb 27, 2026
Merged

feat(motions): implement search_dokument_fulltext, analyze_g0v_by_department, and search_anforanden#603
pethers merged 4 commits intomainfrom
copilot/add-full-text-analysis-tools

Conversation

Copy link
Contributor

Copilot AI commented Feb 26, 2026

  • Update REQUIRED_TOOLS in scripts/news-types/motions.ts to include all 4 tools
  • Add search_dokument_fulltext call with graceful error handling
  • Add analyze_g0v_by_department call with graceful error handling
  • Add search_anforanden call with graceful error handling
  • Attach fullText results to motions for "Policy Alternative" rendering
  • Attach speeches results to motions for "Party Positioning" rendering
  • Add govDeptData optional field to ArticleContentData in scripts/data-transformers/types.ts
  • Add "Government Engagement" section to generateMotionsContent in scripts/data-transformers/content-generators.ts
  • Add govEngagement label to all 14 languages in scripts/data-transformers/constants.ts
  • Add govEngagement to ContentLabelSet interface in scripts/types/content.ts
  • Update tests/news-types/motions.test.ts mock to include request and searchSpeeches
  • Add test for analyze_g0v_by_department graceful degradation
  • Replace hardcoded '2025/26' riksmöte with getCurrentRiksmote() utility function
  • Record failed tool calls in mcpCalls with empty result in all three catch blocks (comments #2862867925, #2862867944, #2862867954)
  • Fix index-based fullText assignment — attach only to primary motion used for query (comment #2862867967)
  • Fix deptCount truthy check to !== null && !== undefined so 0 renders correctly (comment #2862867980)
  • Add mcpCalls assertions to all three graceful-degradation tests (comment #2862867990)
  • All 18 tests pass, TypeScript compiles clean
Original prompt

This section details on the original issue you should resolve

<issue_title>Improve Motions: Add full-text analysis, government response tracking, and debate context</issue_title>
<issue_description>## 📋 Issue Type
Enhancement — Implement missing MCP tools for opposition motions: full-text analysis, government response tracking, and debate context

🎯 Objective

Implement the three TODO MCP tools in scripts/news-types/motions.ts: search_dokument_fulltext (full motion text analysis), analyze_g0v_by_department (government department response tracking), and search_anforanden (debate context and party positioning). These tools will transform motion articles from basic listings into strategic opposition intelligence.

📊 Current State

  • File: scripts/news-types/motions.ts (403 lines)
  • Implemented tools: get_motioner ✅ (only tool)
  • Missing tools: search_dokument_fulltext ❌, analyze_g0v_by_department ❌, search_anforanden
  • REQUIRED_TOOLS: Only 1 tool (get_motioner) — reduced from original 4 to avoid validation failures
  • Impact: Motion articles list opposition proposals without explaining what they actually propose, how government responds, or how they're debated

🚀 Desired State

  • Full-text analysis extracts specific policy alternatives being proposed
  • Government department response tracking shows which proposals get engagement
  • Debate context captures party positioning and cross-party support signals
  • Articles explain opposition strategy (coordinated campaigns, emerging alliances)
  • Cross-party motion co-sponsorship analysis (which parties collaborate)
  • "Adoption potential" scoring based on historical patterns
  • Comparison with government propositions on same topics (alternative framing)

🔧 Implementation Approach

1. Add search_dokument_fulltext integration (~80 lines)

// Extract policy alternatives from motion full text
const fullText = await mcpClient.call('search_dokument_fulltext', {
  query: motion.titel,
  limit: 3
});
  • Parse specific policy alternatives proposed
  • Identify budget implications and funding sources
  • Compare with existing government policy
  • Use scripts/data-transformers/document-analysis.ts for opposition strategy detection

2. Add analyze_g0v_by_department integration (~60 lines)

// Track government department engagement with motion topics
const govResponse = await mcpClient.call('analyze_g0v_by_department', {
  dateFrom: motionDate,
  dateTo: currentDate
});
  • Map motions to responsible government departments
  • Track whether government has responded to motion topics
  • Identify areas where opposition is ahead of government agenda

3. Add search_anforanden integration (~60 lines)

// Find debate speeches related to this motion topic
const debates = await mcpClient.call('search_anforanden', {
  text: motion.titel,
  rm: currentRiksmote,
  limit: 10
});
  • Extract party positions from debate speeches
  • Identify cross-party support signals
  • Capture government response quotes
  • Detect coalition fault lines on motion topics

4. Update REQUIRED_TOOLS

Restore full 4-tool list: get_motioner, search_dokument_fulltext, analyze_g0v_by_department, search_anforanden

5. Enhanced article templates

  • Add "Policy Alternative" section explaining what's proposed
  • Add "Government Engagement" tracking
  • Add "Party Positioning" from debates
  • Add "Cross-Party Support" signals
  • Update all 14 language templates

🤖 Recommended Agent

code-quality-engineer — TypeScript implementation matching existing MCP integration patterns

✅ Acceptance Criteria

  • search_dokument_fulltext integrated for policy alternative extraction
  • analyze_g0v_by_department integrated for government response tracking
  • search_anforanden integrated for debate context and party positioning
  • REQUIRED_TOOLS restored to full 4-tool specification
  • REQUIRED_TOOLS validation passes without warnings
  • Article templates include "Policy Alternative", "Government Response", "Party Positioning" sections
  • All 14 language templates updated
  • Opposition strategy analysis leverages document-analysis.ts
  • No regression in existing motion generation
  • Performance within 20-second batch target for all languages
  • Error handling for tools that return empty results (graceful degradation)

📚 References

  • File: scripts/news-types/motions.ts:57-60 (TODO comments)
  • File: scripts/news-types/motions.ts:189-196 (REQUIRED_TOOLS update comment)
  • MCP Tools: search_dokument_fulltext, analyze_g0v_by_department, search_anforanden
  • Document analysis: scripts/data-transformers/document-analysis.ts (opposition strategy detection)
  • Policy analysis: scripts/data-transformers/policy-analysis.ts
  • Pattern reference: scripts/news-types/weekly-review.ts (full enrichment pipeline example)
    </issue_description>

<agent_i...


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

…search_anforanden in motions.ts

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Copilot AI changed the title [WIP] Improve motions with full-text analysis and response tracking feat(motions): implement search_dokument_fulltext, analyze_g0v_by_department, and search_anforanden Feb 26, 2026
@pethers pethers requested a review from Copilot February 27, 2026 02:12
@github-actions github-actions bot added testing Test coverage refactor Code refactoring size-m Medium change (50-250 lines) labels Feb 27, 2026
@github-actions
Copy link
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements three missing MCP tools in the motions article generator to transform basic motion listings into comprehensive opposition intelligence with full-text analysis, government response tracking, and debate context.

Changes:

  • Restored REQUIRED_TOOLS from 1 to 4 tools with graceful degradation on failures
  • Added three MCP enrichment calls: search_dokument_fulltext, analyze_g0v_by_department, and search_anforanden
  • Extended content generation with "Government Department Engagement" section and enhanced cross-references

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
scripts/news-types/motions.ts Implemented three MCP tools with try-catch error handling, updated REQUIRED_TOOLS to 4, expanded crossReferences sources
scripts/data-transformers/content-generators.ts Added Government Department Engagement section with HTML-escaped output
scripts/data-transformers/types.ts Added govDeptData field to ArticleContentData interface
scripts/types/content.ts Added govEngagement field to ContentLabelSet interface
scripts/data-transformers/constants.ts Added govEngagement label translations for all 14 languages
tests/news-types/motions.test.ts Extended mock client with request and searchSpeeches, added 8 new tests for tool calls and graceful degradation
Comments suppressed due to low confidence (1)

scripts/news-types/motions.ts:333

  • The generateSources function in scripts/data-transformers/metadata.ts doesn't have mappings for the three new tools: search_dokument_fulltext, analyze_g0v_by_department, and search_anforanden. While the tools are correctly called and their results used in article generation, the article footer's "Data Sources" section won't include human-readable names for these enrichment sources. Consider adding mappings like: "Full-text Document Analysis", "Government Department Tracking", and "Parliamentary Debate Speeches" to improve transparency about data sources used in the intelligence analysis.
      const sources: string[] = generateSources([
        'get_motioner',
        'search_dokument_fulltext',
        'analyze_g0v_by_department',
        'search_anforanden',
      ]);

mockClientInstance.searchSpeeches.mockRejectedValueOnce(new Error('Tool unavailable'));
const result = await motionsModule.generateMotions({ languages: ['en'] });
expect(result.success).toBe(true);
});
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing test coverage for analyze_g0v_by_department graceful degradation. The implementation calls client.request() twice (for search_dokument_fulltext at line 257 and analyze_g0v_by_department at line 279), but there's no test verifying that failure of the second tool call is handled gracefully. The test at line 160 only covers the first request call failure using mockRejectedValueOnce. Add a test that specifically verifies analyze_g0v_by_department failure doesn't break article generation.

Suggested change
});
});
it('should degrade gracefully when analyze_g0v_by_department fails', async () => {
const originalRequestImpl = mockClientInstance.request.getMockImplementation();
// First call (search_dokument_fulltext) uses the existing implementation
mockClientInstance.request.mockImplementationOnce(
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(...args: any[]) => originalRequestImpl && originalRequestImpl(...args)
);
// Second call (analyze_g0v_by_department) fails
mockClientInstance.request.mockRejectedValueOnce(new Error('Tool unavailable'));
const result = await motionsModule.generateMotions({ languages: ['en'] });
expect(result.success).toBe(true);
});

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a dedicated test 'should degrade gracefully when analyze_g0v_by_department fails' in commit 1ef7492. It uses mockResolvedValueOnce({}) for the first request call (search_dokument_fulltext) and mockRejectedValueOnce(new Error(...)) for the second (analyze_g0v_by_department), confirming the article still generates successfully when that tool fails.

try {
const debateQuery = motions[0]?.titel || motions[0]?.title || '';
if (debateQuery) {
const speeches = await client.searchSpeeches({ text: debateQuery, rm: '2025/26', limit: 10 }) as Array<Record<string, unknown>>;
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoded riksmöte value '2025/26' will become outdated when the parliamentary year changes (typically September/October). The Swedish riksmöte follows the pattern YYYY/YY where the year changes each September. Consider extracting a utility function to calculate the current riksmöte dynamically based on the current date, or at minimum document when this value needs to be updated. This same hardcoded value appears in monthly-review.ts and weekly-review.ts, indicating a broader maintainability concern across the codebase.

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit 1ef7492. Added getCurrentRiksmote(date) utility that computes the session string dynamically (month >= 8 ? year : year - 1, since getMonth() is 0-based so September = 8). The search_anforanden call now uses getCurrentRiksmote() instead of the hardcoded '2025/26'.

@pethers
Copy link
Member

pethers commented Feb 27, 2026

@copilot apply changes based on the comments in this thread

…öte calculation

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

});
}
} catch (err) {
console.warn(' ⚠ search_dokument_fulltext unavailable:', (err as Error).message);
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the search_dokument_fulltext catch block, the tool failure is swallowed without recording the tool in mcpCalls. This can cause cross-reference validation to report missing required tools, and the article can still list the tool in sources even though it didn’t produce data. Record the attempted call in mcpCalls with an empty result (and optionally an error marker) when the request fails, or derive sources from the actual mcpCalls list.

Suggested change
console.warn(' ⚠ search_dokument_fulltext unavailable:', (err as Error).message);
console.warn(' ⚠ search_dokument_fulltext unavailable:', (err as Error).message);
// Record attempted tool call with empty result so validation and sources stay consistent
mcpCalls.push({ tool: 'search_dokument_fulltext', result: [] });

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ca4ea6f. The catch block now pushes { tool: 'search_dokument_fulltext', result: [] } to mcpCalls so validation and source attribution stay consistent even on failure.

mcpCalls.push({ tool: 'analyze_g0v_by_department', result: govDeptData });
console.log(` 🏛 Gov dept analysis: ${govDeptData.length} departments`);
} catch (err) {
console.warn(' ⚠ analyze_g0v_by_department unavailable:', (err as Error).message);
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

analyze_g0v_by_department failures currently only log a warning and don’t add an entry to mcpCalls. Since this tool is in REQUIRED_TOOLS, missing the record can fail cross-reference validation even though generation returns success. Add a fallback mcpCalls entry (empty array/result + optional error info) in the catch path so downstream validation and reporting remain consistent.

Suggested change
console.warn(' ⚠ analyze_g0v_by_department unavailable:', (err as Error).message);
const error = err as Error;
console.warn(' ⚠ analyze_g0v_by_department unavailable:', error.message);
// Fallback MCP call record to keep REQUIRED_TOOLS cross-reference consistent
mcpCalls.push({
tool: 'analyze_g0v_by_department',
result: [],
error: {
message: error.message,
name: error.name,
},
} as MCPCallRecord);

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ca4ea6f. The catch block now pushes { tool: 'analyze_g0v_by_department', result: [] } to keep mcpCalls consistent with REQUIRED_TOOLS.

}
}
} catch (err) {
console.warn(' ⚠ search_anforanden unavailable:', (err as Error).message);
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

search_anforanden failures are not recorded in mcpCalls, but the tool remains required and listed as a source. To keep validation and source attribution accurate during graceful degradation, add an mcpCalls entry in the catch path (e.g., empty speeches array and/or captured error message).

Suggested change
console.warn(' ⚠ search_anforanden unavailable:', (err as Error).message);
console.warn(' ⚠ search_anforanden unavailable:', (err as Error).message);
mcpCalls.push({ tool: 'search_anforanden', result: [] });

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ca4ea6f. The catch block now pushes { tool: 'search_anforanden', result: [] } to mcpCalls.

Comment on lines 274 to 279
ftDocs.forEach((ftDoc, i) => {
const m = motions[i] as Record<string, unknown>;
if (m && !m['fullText']) {
m['fullText'] = ftDoc.fullText || ftDoc.summary || '';
}
});
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

search_dokument_fulltext is queried once using only the first motion’s title, but the results are then assigned to motions by index and written into each motion’s fullText field. This can easily attach unrelated text to other motions and pollute downstream summaries/policy analysis. Consider keeping these documents as a separate related-docs collection, or do per-motion lookup / match on dok_id (when available) rather than index-based assignment.

Suggested change
ftDocs.forEach((ftDoc, i) => {
const m = motions[i] as Record<string, unknown>;
if (m && !m['fullText']) {
m['fullText'] = ftDoc.fullText || ftDoc.summary || '';
}
});
const primaryMotion = motions[0] as Record<string, unknown> | undefined;
if (primaryMotion && ftDocs.length > 0) {
// Attach the most relevant full text only to the motion that was used for the query
if (!primaryMotion['fullText']) {
const bestDoc = ftDocs[0] as Record<string, unknown>;
primaryMotion['fullText'] = (bestDoc['fullText'] as string) || (bestDoc['summary'] as string) || '';
}
// Optionally expose all related documents for downstream "Policy Alternative" rendering
if (!primaryMotion['policyAlternativeDocs']) {
primaryMotion['policyAlternativeDocs'] = ftDocs;
}
}

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ca4ea6f. Replaced the forEach index loop with a targeted assignment: only motions[0] (the motion used for the query) receives fullText from ftDocs[0]. All related docs are exposed on primaryMotion['policyAlternativeDocs'] when multiple results are returned.

Comment on lines 785 to 786
if (deptName) {
content += deptCount
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deptCount is checked using a truthy test, so a legitimate value of 0 will be treated as “missing” and won’t render in parentheses. Prefer checking for null/undefined (e.g., deptCount !== null && deptCount !== undefined) so 0 still displays correctly.

Suggested change
if (deptName) {
content += deptCount
const hasDeptCount = deptCount !== null && deptCount !== undefined;
if (deptName) {
content += hasDeptCount

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ca4ea6f. Changed to const hasDeptCount = deptCount !== null && deptCount !== undefined so a value of 0 renders correctly in parentheses.

Comment on lines 160 to 173
it('should degrade gracefully when search_dokument_fulltext fails', async () => {
mockClientInstance.request.mockRejectedValueOnce(new Error('Tool unavailable'));
const result = await motionsModule.generateMotions({ languages: ['en'] });
expect(result.success).toBe(true);
});

it('should degrade gracefully when analyze_g0v_by_department fails', async () => {
// First request call (search_dokument_fulltext) succeeds; second (analyze_g0v_by_department) fails
mockClientInstance.request
.mockResolvedValueOnce({})
.mockRejectedValueOnce(new Error('Tool unavailable'));
const result = await motionsModule.generateMotions({ languages: ['en'] });
expect(result.success).toBe(true);
});
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current tests assert generation succeeds when enrichment tools fail, but they don’t verify that the failed tool is still recorded in mcpCalls (which downstream cross-reference validation relies on). Add assertions in the graceful-degradation tests to ensure mcpCalls includes the failed tool name with an empty result/error marker.

Copilot generated this review using guidance from repository custom instructions.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ca4ea6f. All three graceful-degradation tests now assert result.mcpCalls?.map(c => c.tool) contains the failed tool name, verifying the empty-result fallback entry is recorded.

@pethers
Copy link
Member

pethers commented Feb 27, 2026

@copilot apply changes based on the comments in this thread

…ignment, fix deptCount null check

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Signed-off-by: James Pether Sörling <pethers@users.noreply.github.com>
@github-actions
Copy link
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

scripts/data-transformers/types.ts:101

  • The ArticleContentData interface now has two fields for essentially the same data from analyze_g0v_by_department:
  • govDeptData (line 97): An array of department records, used by motions.ts
  • departmentAnalysis (line 101): A single object containing a 'departments' property, used by propositions.ts

This creates an inconsistent API where different news types use different field names and shapes for the same MCP tool response. Consider standardizing on one field name and shape across all news types. For example, consolidate to use govDeptData as the direct array (since it's simpler), and update propositions.ts to extract the array from the response before passing it to generateArticleContent.

  /** Government department analysis from analyze_g0v_by_department */
  govDeptData?: Record<string, unknown>[];
  /** Full-text search results for policy substance extraction */
  fullTextResults?: unknown[];
  /** Government department analysis from analyze_g0v_by_department */
  departmentAnalysis?: Record<string, unknown>;

@pethers pethers marked this pull request as ready for review February 27, 2026 10:15
@pethers pethers merged commit ab4eb79 into main Feb 27, 2026
19 checks passed
@pethers pethers deleted the copilot/add-full-text-analysis-tools branch February 27, 2026 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

refactor Code refactoring size-m Medium change (50-250 lines) testing Test coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve Motions: Add full-text analysis, government response tracking, and debate context

3 participants