[Repo Assist] docs(roadmap): add W3 requirements and traceability status analysis by github-actions[bot] · Pull Request #353 · informedica/GenPRES

github-actions · 2026-05-17T21:14:08Z

🤖 This is an automated PR from Repo Assist.

Summary

Adds docs/roadmap/w3-requirements-and-traceability.md — a status analysis for the W3: Requirements & Traceability workshop, following the same pattern as the W2 review document (PR #351).

Change

The document covers the four W3 goals from the ROADMAP:

Goal	Status
Requirements review and validation	🔄 Partial
Traceability matrix completion	🔄 Partial
Test coverage analysis	✅ Complete
Gap identification	🔄 Partial

What's documented as complete:

software-requirements.md v1.1 (May 2026) — updated to reflect .NET 10, FHIR R4, MCP, NLP, security baseline
user-requirements.md with UR-001–020
Full test validation baseline: test-strategy.md, unit-test-report.md (≥ 5 408 tests), integration-test-report.md (32 integration tests)
ADR reference table (0000–0020) linking design decisions

What's documented as remaining:

Markdown traceability matrix (UR → SR → ADR → test case linkage)
Formal gap analysis document consolidating known coverage gaps
Clinical expert review (deferred to W4)

Overall W3 status: ~60–70% complete.

Test Status

This PR contains only documentation — no source code changes. No build or test steps required.

Repo Assist — automated PR · run 26002674784

Generated by Repo Assist

To install this agentic workflow, run
gh aw add githubnext/agentics/workflows/repo-assist.md@f2c5cf1e4af58e09a93ba0703c6bf084711b265f

Adds docs/roadmap/w3-requirements-and-traceability.md tracking progress toward the W3 workshop goals: requirements review and validation (partial - software-requirements.md v1.1, user-requirements.md UR-001-020), traceability matrix (partial - Excel matrices exist, markdown linkage pending), test coverage analysis (complete - test-strategy.md, unit-test-report.md, integration-test-report.md, 5408+ tests), and gap identification (partial - known gaps scattered, formal doc pending). Includes ADR reference table (0000-0020), remaining tasks checklist, and a W3 ~60-70% completion summary. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

greptile-apps · 2026-05-18T05:27:59Z

Greptile Summary

Adds docs/roadmap/w3-requirements-and-traceability.md, an automated status analysis of the W3 Requirements & Traceability workshop goals, following the same pattern as the W2 review. The document is documentation-only and touches no source code.

Covers the four W3 ROADMAP goals (requirements review, traceability matrix, test coverage, gap identification), marking three as partial and one as complete.
Several gap claims in the "Remaining / In Progress" section are directly contradicted by unit-test-report.md: Informedica.FHIR.Tests and Informedica.MCP.Tests are both listed as passing CI test projects, and Informedica.NLP.Tests covers LLM-independent NLP logic — making those portrayed gaps either closed or overstated.

Confidence Score: 3/5

The document is documentation-only, but it contains factual inaccuracies about test coverage that could give maintainers a misleading picture of what still needs to be done before the W3 milestone is closed.

The gap claims for FHIR R4 and MCP tests are directly contradicted by the unit test report in the same repository — both test projects are already in CI. If taken at face value, the document would lead the team to re-do work that is already complete. The NLP description also understates existing coverage.

docs/roadmap/w3-requirements-and-traceability.md — the Remaining / In Progress section should be reconciled against docs/mdr/validation/unit-test-report.md before merging.

Important Files Changed

Filename	Overview
docs/roadmap/w3-requirements-and-traceability.md	New W3 status document; several factual claims about test coverage gaps (FHIR, MCP, NLP) are contradicted by the referenced unit-test-report.md, and the GenSOLVER scripts row contains unsupported detail.

_{Reviews (1): Last reviewed commit: "docs(roadmap): add W3 requirements and t..." | Re-trigger Greptile}

greptile-apps · 2026-05-18T05:28:03Z

+- [ ] Draft `docs/mdr/requirements/gap-analysis.md` consolidating the above
+      gaps with proposed resolution paths.
+- [ ] Identify which gaps are acceptable for v2.0 release (deferred risk) vs.


FHIR R4 gap claim contradicts unit test report

docs/mdr/validation/unit-test-report.md §4.7 explicitly lists Informedica.FHIR.Tests with "FHIR R4 MedicationRequest translation tests (ADR-0020)" as one of the 20 test projects executed via dotnet run ServerTests. The unit test report introduction states all tests are run via that command, so characterising the FHIR tests as a prototype outside CI is no longer accurate — this gap appears to have been closed already.

greptile-apps · 2026-05-18T05:28:04Z

+- [ ] Draft `docs/mdr/requirements/gap-analysis.md` consolidating the above
+      gaps with proposed resolution paths.
+- [ ] Identify which gaps are acceptable for v2.0 release (deferred risk) vs.
+      must-fix before clinical deployment.


MCP gap claim contradicted by unit test report

docs/mdr/validation/unit-test-report.md §4.7 lists Informedica.MCP.Tests with "MCP stdio server tool registration and dispatch" as part of the dotnet run ServerTests suite. Stating there are "no dedicated integration tests" for MCP command handlers is inaccurate; the test project exists and passes in CI.

greptile-apps · 2026-05-18T05:28:05Z

+Documents the per-module test coverage baseline:
+
+| Module | Coverage Highlights |
+|--------|---------------------|


GenSOLVER scripts row description is unsupported

The unit test report's GenSOLVER section describes Scripts/Tests.fsx only as "F# Interactive coverage for running and validating GenSOLVER test cases during script-based development." There is no mention of "Canon-key/LRU/cycle-detection" tests in that section — this specific claim doesn't appear to be drawn from the referenced source document.

greptile-apps · 2026-05-18T05:28:06Z

+  automated test verifies log output format or completeness.
+
+**W3 remaining tasks:**


NLP gap overstates the situation

unit-test-report.md §4.7 lists Informedica.NLP.Tests with "NLP pipeline unit tests (LLM-independent portions)" as a passing CI test project. The gap for the LLM-dependent .fsx scripts is real (noted in §5 of that report), but saying the NLP pipeline has "no formal test suite beyond the .fsx prototype" ignores the existing Informedica.NLP.Tests project.

Copilot

Pull request overview

Adds a W3 roadmap status analysis document for requirements and traceability, summarizing requirement documents, validation/test reports, ADR coverage, and remaining traceability/gap-analysis work.

Changes:

Adds W3 goal status table and completion summary.
Summarizes requirements, test validation reports, and ADR references.
Lists remaining traceability matrix, gap analysis, and clinical review tasks.

+This document analyses progress toward the W3 workshop goals as defined in
+the [GenPRES Architecture and Timeline](genpres-architecture-and-timeline.md)
+and [ROADMAP](../../ROADMAP.md).


+- 20 test projects under `tests/`, each covering a dedicated library.
+- Expecto 10.x test runner with FsCheck property-based testing.
+- CI matrix across Ubuntu, Windows, and macOS (GitHub Actions `build.yml`).


+| GenSOLVER (scripts) | Canon-key/LRU/cycle-detection tests in `.fsx` |
+
+**Baseline total**: ≥ 5 408 passing tests (0 failures, 0 skipped) as of May 2026.


+**Requirement coverage gaps (from `unit-test-report.md`):**
+- Client-side Fable/Elmish code is covered by manual exploratory testing only;
+  no automated component-level tests exist.
+- NLP dose-rule extraction pipeline (`DoseRuleExtract.fsx`) has no formal
+  test suite beyond the `.fsx` prototype.
+- FHIR R4 translation (`FhirExpectoTests.fsx`) has prototype Expecto tests
+  but is not yet part of the CI `dotnet run ServerTests` suite.
+- MCP server command handlers have no dedicated integration tests.


+---
+
+*This document was generated by Repo Assist and reflects the state of the
+`master` branch as of 2026-05-17. It is intended as a progress aide for the


halcwb · 2026-05-18T05:46:05Z

@copilot apply changes based on the comments in this thread

…y.md - Remove incorrect architecture/timeline doc reference as W3 goal source; only ROADMAP.md defines W3 goals - Clarify that FHIR, NLP, and MCP test projects have placeholder tests only - Mark GenSOLVER .fsx scripts as manual/script-only, not CI baseline - Fix mixed source citations in gap analysis and add missing formal line/branch coverage metrics gap - Fix typo: 'progress aide' -> 'progress aid' Agent-Logs-Url: https://github.com/informedica/GenPRES/sessions/e4b86995-65ee-4c13-9840-c32f6fdd4481 Co-authored-by: halcwb <683631+halcwb@users.noreply.github.com>

Copilot · 2026-05-18T05:48:04Z

@copilot apply changes based on the comments in this thread

Applied all 5 review corrections in ad000b2:

Lines 7-9 — Removed the architecture/timeline doc as a source for W3 goals; the intro now cites only ROADMAP.md.
Lines 62-64 — Changed "each covering a dedicated library" to "the majority covering a dedicated library" with an explicit note that FHIR, NLP, and MCP test projects contain only placeholder Hello World tests.
Lines 78-80 — GenSOLVER .fsx scripts row now reads "manual/script-only; not part of dotnet run ServerTests CI baseline".
Lines 141-148 — Gap section heading updated to cite both unit-test-report.md and integration-test-report.md; FHIR/MCP bullets carry (source: integration-test-report.md) attribution; added the missing formal line/branch coverage metrics gap flagged by both test-strategy.md and unit-test-report.md.
Line 201 — Fixed typo: "progress aide" → "progress aid".

github-actions Bot added automation repo-assist labels May 17, 2026

halcwb marked this pull request as ready for review May 18, 2026 05:25

greptile-apps Bot reviewed May 18, 2026

View reviewed changes

halcwb requested a review from Copilot May 18, 2026 05:36

Copilot started reviewing on behalf of halcwb May 18, 2026 05:36 View session

Copilot AI reviewed May 18, 2026

View reviewed changes

Copilot started work on behalf of halcwb May 18, 2026 05:46 View session

Copilot AI requested a review from halcwb May 18, 2026 05:49

Copilot finished work on behalf of halcwb May 18, 2026 05:49

halcwb merged commit b7089a2 into master May 18, 2026
4 checks passed

halcwb deleted the repo-assist/docs-w3-requirements-traceability-17may2026-6769d4c7cb2f52a8 branch May 18, 2026 05:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Repo Assist] docs(roadmap): add W3 requirements and traceability status analysis#353

[Repo Assist] docs(roadmap): add W3 requirements and traceability status analysis#353
halcwb merged 2 commits into
masterfrom
repo-assist/docs-w3-requirements-traceability-17may2026-6769d4c7cb2f52a8

github-actions Bot commented May 17, 2026

Uh oh!

greptile-apps Bot commented May 18, 2026

Uh oh!

greptile-apps Bot May 18, 2026

Uh oh!

greptile-apps Bot May 18, 2026

Uh oh!

greptile-apps Bot May 18, 2026

Uh oh!

greptile-apps Bot May 18, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

halcwb commented May 18, 2026

Uh oh!

Copilot AI commented May 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		automated test verifies log output format or completeness.

		W3 remaining tasks:

		\| GenSOLVER (scripts) \| Canon-key/LRU/cycle-detection tests in `.fsx` \|

		Baseline total: ≥ 5 408 passing tests (0 failures, 0 skipped) as of May 2026.

Conversation

github-actions Bot commented May 17, 2026

Summary

Change

Test Status

Uh oh!

greptile-apps Bot commented May 18, 2026

Greptile Summary

Confidence Score: 3/5

Important Files Changed

Uh oh!

greptile-apps Bot May 18, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 18, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 18, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

halcwb commented May 18, 2026

Uh oh!

Copilot AI commented May 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants