Feature/ab#32009 prompt version baselines by jacobwillsmith · Pull Request #2105 · bcgov/Unity

jacobwillsmith · 2026-03-07T02:55:02Z

Pull request overview

This pull request finalizes runtime prompt versioning for AI generation using file-based v0/v1 templates and a tokenized resolver in OpenAIService. It aligns the active v1 prompt contracts with the typed AI DTO flow and keeps v0 available for controlled legacy behavior.

It also standardizes payload construction across analysis and scoresheet generation by introducing shared submission-shape normalization and a shared fallback payload model. This reduces divergence between prompt paths and keeps orchestration behavior predictable.

Additional correctness and maintainability updates from this branch are preserved, including async authorization loop fixes, centralized text extraction routing, and Sonar-driven cleanup in AI orchestration/services.

Changes:

Added file-driven prompt version architecture under AI/Prompts/Versions with runtime version selection (v0/v1)
Implemented tokenized prompt composition in OpenAIService (including shared/common rule fragments)
Aligned AI service contracts and DTO usage with typed request/response flow
Standardized analysis/scoresheet submission parsing through shared normalization helpers in GenerateAIContentHandler
Standardized analysis/scoresheet fallback payload shape for internal consistency
Preserved v0 compatibility path while keeping v1 as canonical active contract
Kept prior branch fixes: async action authorization correction and extractor dispatch consolidation

…rofiles

… dispatch

github-actions · 2026-03-07T02:58:38Z

🧪 Unit Test Results (Parallel Execution)

📊 Summary

Result	Count
✅ Passed	`465`
❌ Failed	`0`
⚠️ Skipped	`0`

📄 HTML Reports

Merged Tests (HTML): Included in artifacts
Generated automatically by CI.

github-actions · 2026-03-10T19:03:39Z

🧪 Unit Test Results (Parallel Execution)

📊 Summary

Result	Count
✅ Passed	`466`
❌ Failed	`0`
⚠️ Skipped	`0`

📄 HTML Reports

Merged Tests (HTML): Included in artifacts
Generated automatically by CI.

github-actions · 2026-03-10T19:20:08Z

🧪 Unit Test Results (Parallel Execution)

📊 Summary

Result	Count
✅ Passed	`466`
❌ Failed	`0`
⚠️ Skipped	`0`

📄 HTML Reports

Merged Tests (HTML): Included in artifacts
Generated automatically by CI.

jacobwillsmith · 2026-03-10T19:33:48Z

Will move these prompt templates/files to database in a following story

Copilot

Pull request overview

Finalizes runtime prompt versioning for AI generation by moving prompts into file-based templates (v0/v1) resolved at runtime by OpenAIService, and standardizes prompt payload construction/fallbacks to keep analysis and scoresheet flows consistent.

Changes:

Introduces versioned, tokenized prompt templates under AI/Prompts/Versions and adds a runtime resolver/renderer in OpenAIService.
Aligns AI service contracts and orchestration to typed DTOs and normalized submission payloads (analysis + scoresheet).
Preserves additional fixes (e.g., async authorization loop correction, extractor dispatch consolidation).

Reviewed changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
applications/Unity.GrantManager/src/Unity.GrantManager.Application/Unity.GrantManager.Application.csproj	Copies versioned prompt template `.txt` files to build output.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/Intakes/Handlers/GenerateAIContentHandler.cs	Normalizes submission parsing and unifies fallback payload shape for analysis/scoresheet.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/GrantApplications/GrantApplicationAppService.cs	Fixes async authorization loop by replacing `ForEach(async ...)` with `foreach` + `await`.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/TextExtractionService.cs	Centralizes extractor selection via extension-dispatch dictionary and simplifies content-type fallbacks.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/scoresheet.user.txt	Adds v1 scoresheet user template with token placeholders.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/scoresheet.system.txt	Adds v1 scoresheet system template.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/scoresheet.rules.txt	Adds v1 scoresheet rules fragment.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/scoresheet.output.txt	Adds v1 scoresheet output fragment.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/common.rules.txt	Adds shared v1 common rules fragment.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/attachment.user.txt	Adds v1 attachment user template.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/attachment.system.txt	Adds v1 attachment system template.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/attachment.rules.txt	Adds v1 attachment rules fragment.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/attachment.output.txt	Adds v1 attachment output fragment.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/analysis.user.txt	Adds v1 analysis user template (schema/data/attachments + fragment placeholders).
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/analysis.system.txt	Adds v1 analysis system template.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/analysis.score.txt	Adds v1 analysis score fragment.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/analysis.rules.txt	Adds v1 analysis rules fragment.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/analysis.rubric.txt	Adds v1 analysis rubric fragment.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v1/analysis.output.txt	Adds v1 analysis output fragment (typed findings contract).
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v0/scoresheet.user.txt	Adds v0 legacy scoresheet user template (monolithic prompt).
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v0/scoresheet.system.txt	Adds v0 legacy scoresheet system template.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v0/attachment.user.txt	Adds v0 legacy attachment user template.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v0/attachment.system.txt	Adds v0 legacy attachment system template.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v0/analysis.user.txt	Adds v0 legacy analysis user template.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/v0/analysis.system.txt	Adds v0 legacy analysis system template.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/Versions/README.md	Documents template layout, placeholders, fragment resolution rules, and version selection.
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/ScoresheetPrompts.cs	Removes hard-coded scoresheet prompt builder (replaced by file templates).
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/PromptHeader.cs	Removes prompt header helper (replaced by file templates).
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/PromptCoreRules.cs	Removes core rules constants (replaced by common fragment templates).
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/AttachmentPrompts.cs	Removes hard-coded attachment prompt builder (replaced by file templates).
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/Prompts/AnalysisPrompts.cs	Removes hard-coded analysis prompt builder/rubric/output (replaced by file templates).
applications/Unity.GrantManager/src/Unity.GrantManager.Application/AI/OpenAIService.cs	Implements runtime prompt version selection, template loading/caching, fragment rendering, and aligns typed DTO methods.
applications/Unity.GrantManager/src/Unity.GrantManager.Application.Contracts/AI/Requests/ApplicationAnalysisRequest.cs	Removes rubric field from request (rubric now comes from templates/fragments).
applications/Unity.GrantManager/src/Unity.GrantManager.Application.Contracts/AI/Requests/AICompletionRequest.cs	Removes system prompt override and keeps temperature/max tokens configuration.
applications/Unity.GrantManager/src/Unity.GrantManager.Application.Contracts/AI/IAIService.cs	Simplifies interface to typed AI operations and renames scoresheet call to `GenerateScoresheetSectionAsync`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

github-actions · 2026-03-10T21:33:27Z

🧪 Unit Test Results (Parallel Execution)

📊 Summary

Result	Count
✅ Passed	`466`
❌ Failed	`0`
⚠️ Skipped	`0`

📄 HTML Reports

Merged Tests (HTML): Included in artifacts
Generated automatically by CI.

JamesPasta

This is a great first start at moving out the prompts and will help us with creating the data transitions.

jacobwillsmith added 5 commits March 6, 2026 16:11

AB#32009 Add prompt baseline snapshots for v0 and v1 comparison

9667a10

AB#32009 Normalize prompt baseline version folders and snapshots

9ee1136

AB#32009 Add runtime prompt version selector with single-file v0/v1 p…

979e7b9

…rofiles

AB#32009 Align v1 prompt contracts and preserve v0 path

73e791f

AB#32009 Fix async action authorization loop and centralize extractor…

c0e25bd

… dispatch

jacobwillsmith added 3 commits March 10, 2026 08:29

AB#32009 Normalize baseline snapshot filenames to .txt convention

10e8ec7

AB#32009 Normalize versioned AI prompt template architecture

e7f0df1

AB#32009 unify prompt normalization and fallback payloads

c38438b

AB#32009 address remaining Sonar prompt and extractor issues

ec4186a

jacobwillsmith requested review from AndreGAot, JamesPasta, hasanpour and ligovbc March 10, 2026 19:33

jacobwillsmith marked this pull request as ready for review March 10, 2026 19:33

JamesPasta requested a review from Copilot March 10, 2026 20:34

Copilot started reviewing on behalf of JamesPasta March 10, 2026 20:35 View session

Copilot AI reviewed Mar 10, 2026

View reviewed changes

AB#32009 apply copilot follow-ups for prompt loading and section guard

90bd25f

JamesPasta approved these changes Mar 10, 2026

View reviewed changes

jacobwillsmith merged commit 1e317f1 into dev Mar 10, 2026
22 checks passed

jacobwillsmith deleted the feature/AB#32009-PromptVersionBaselines branch March 10, 2026 23:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/ab#32009 prompt version baselines#2105

Feature/ab#32009 prompt version baselines#2105
jacobwillsmith merged 10 commits into
devfrom
feature/AB#32009-PromptVersionBaselines

jacobwillsmith commented Mar 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Mar 7, 2026

Uh oh!

github-actions Bot commented Mar 10, 2026

Uh oh!

github-actions Bot commented Mar 10, 2026

Uh oh!

jacobwillsmith commented Mar 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Mar 10, 2026

Uh oh!

JamesPasta left a comment •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jacobwillsmith commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request overview

Uh oh!

github-actions Bot commented Mar 7, 2026

🧪 Unit Test Results (Parallel Execution)

📊 Summary

📄 HTML Reports

Uh oh!

github-actions Bot commented Mar 10, 2026

🧪 Unit Test Results (Parallel Execution)

📊 Summary

📄 HTML Reports

Uh oh!

github-actions Bot commented Mar 10, 2026

🧪 Unit Test Results (Parallel Execution)

📊 Summary

📄 HTML Reports

Uh oh!

jacobwillsmith commented Mar 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Mar 10, 2026

🧪 Unit Test Results (Parallel Execution)

📊 Summary

📄 HTML Reports

Uh oh!

JamesPasta left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jacobwillsmith commented Mar 7, 2026 •

edited

Loading

JamesPasta left a comment •

edited

Loading