Skip to content

Improve pr-docs-check reliability guidance - #17552

Closed
David Pine (IEvangelist) wants to merge 1 commit into
mainfrom
ievangelist/dapine-pr-docs-failures
Closed

Improve pr-docs-check reliability guidance#17552
David Pine (IEvangelist) wants to merge 1 commit into
mainfrom
ievangelist/dapine-pr-docs-failures

Conversation

@IEvangelist

Copy link
Copy Markdown
Member

Description

This improves reliability of the pr-docs-check agentic workflow by tightening prompt guidance around known safe-output failure modes, while keeping core workflow mechanics (auth, checkout, and branch-resolution flow) unchanged.

Key improvements:

  • Add hard reliability guardrails for generated docs edits so the agent avoids broad/high-risk doc patches and handles oversized change sets through report_incomplete.
  • Add explicit locked-source-PR handling so notification output does not attempt comment posting when the source PR is locked.
  • Recompile workflow artifacts to keep the lock files in sync with source guidance updates.

Validation:

  • gh aw compile .github/workflows/pr-docs-check.md completed successfully with no errors/warnings.

Fixes # (issue)

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No

Add explicit guardrails in the pr-docs-check workflow prompt to reduce deterministic safe-output failures, then recompile workflow artifacts.

- Add constraints to keep generated docs patches targeted and under safe-output limits.
- Add explicit handling for locked source PRs to avoid hard-failing notify flow.
- Regenerate lock artifacts via gh aw compile.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@IEvangelist
David Pine (IEvangelist) marked this pull request as ready for review May 27, 2026 18:08
Copilot AI review requested due to automatic review settings May 27, 2026 18:08
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 17552

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 17552"

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR tightens the pr-docs-check agentic workflow prompt with reliability guardrails (limit doc patch size, fall back to report_incomplete on risky changes, and skip notify_source_pr when the source PR is locked) and recompiles the lock files. The compiled .lock.yml was regenerated with gh-aw v0.74.8, bringing along upstream toolchain bumps (firewall 0.25.49, mcpg v0.3.9, github-mcp-server v1.0.4, copilot CLI 1.0.48, model default claude-sonnet-4.5).

Changes:

  • Add "Reliability guardrails for generated doc edits" and "Locked source PR handling" sections to pr-docs-check.md.
  • Recompile pr-docs-check.lock.yml with gh-aw v0.74.8 (action/container pins, span-id propagation, DOCKER_HOST handling, token-steering AWF config).
  • Update actions-lock.json to pin github/gh-aw-actions/setup@v0.74.8 and drop the unused setup-cli entry.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
.github/workflows/pr-docs-check.md Adds hard reliability guardrails and locked-PR handling guidance to the workflow prompt.
.github/workflows/pr-docs-check.lock.yml Regenerated by gh aw compile with v0.74.8 toolchain bumps.
.github/aw/actions-lock.json Bumps gh-aw-actions/setup pin to v0.74.8; removes stale setup-cli entry.

@github-actions

Copy link
Copy Markdown
Contributor

CLI E2E Tests failed — 106 passed, 1 failed, 2 unknown (commit 40387de)

❌ Failed Tests

Status Test Recording Job Artifacts
UpdateProjectChannelToStable_TypeScript_PreviewsStablePackagesAndPreservesChannel Recording #78302196555 Logs
View all recordings
Status Test Recording Job Artifacts
AddPackageInteractiveWhileAppHostRunningDetached Recording #78302196188 Logs
AddPackageWhileAppHostRunningDetached Recording #78302196188 Logs
AgentCommands_AllHelpOutputs_AreCorrect Recording #78302197246 Logs
AgentInitCommand_DefaultSelection_InstallsDefaultSkills Recording #78302197246 Logs
AgentInitCommand_MigratesDeprecatedConfig Recording #78302197246 Logs
AgentMcpListStructuredLogsReturnsLogsFromStarterApp Recording #78302197154 Logs
AgentMcpListStructuredLogsReturnsLogsFromStarterApp_DevLocalhost Recording #78302197154 Logs
AgentMcpListStructuredLogsReturnsLogsFromStarterApp_Isolated Recording #78302197154 Logs
AllPublishMethodsBuildDockerImages Recording #78302197362 Logs
AspireAddAndStartWorkAgainstLegacyAppHostTs Recording #78302197878 Logs
AspireAddPackageVersionToDirectoryPackagesProps Recording #78302197108 Logs
AspireInitSingleFileAppHostRunsViaDotnetRunAppHost Recording #78302197558 Logs
AspireInitWithExistingAppHostDirRecreatesMissingNuGetConfigAndPreservesFiles Recording #78302197045 Logs
AspireInitWithSolutionFileGeneratesAppHostThatBuildsAgainstChannelHive Recording #78302197045 Logs
AspireStartUpdatesStaleTypeScriptAppHostPath Recording #78302198162 Logs
AspireUpdateRemovesAppHostPackageVersionFromDirectoryPackagesProps Recording #78302197108 Logs
AspireUpdateRemovesOrphanAppHostPackageVersionWhenSdkAlreadyCurrent Recording #78302197108 Logs
Banner_DisplayedOnFirstRun Recording #78302197231 Logs
Banner_DisplayedWithExplicitFlag Recording #78302197231 Logs
Banner_NotDisplayedWithNoLogoFlag Recording #78302197231 Logs
CertificatesClean_RemovesCertificates Recording #78302196944 Logs
CertificatesTrust_WithNoCert_CreatesAndTrustsCertificate Recording #78302196944 Logs
CertificatesTrust_WithUntrustedCert_TrustsCertificate Recording #78302196944 Logs
ConfigSetGet_CreatesNestedJsonFormat Recording #78302197776 Logs
CreateAndRunAspireStarterProject Recording #78302198336 Logs
CreateAndRunAspireStarterProjectWithBundle Recording #78302196804 Logs
CreateAndRunEmptyAppHostProject Recording #78302197653 Logs
CreateAndRunJavaEmptyAppHostProject Recording #78302196916 Logs
CreateAndRunJsReactProject Recording #78302196859 Logs
CreateAndRunPythonReactProject Recording #78302197052 Logs
CreateAndRunTypeScriptEmptyAppHostProject Recording #78302197497 Logs
CreateAndRunTypeScriptStarterProject Recording #78302197167 Logs
CreateJavaAppHostWithViteApp Recording #78302196240 Logs
CreateTypeScriptAppHostWithViteApp_AllowsGuestAppPackageManagerToDiffer Recording #78302197605 Logs
CreateTypeScriptAppHostWithViteApp_UsesConfiguredToolchain Recording #78302197605 Logs
DashboardRunWithAgentMcpListTracesReturnsNoTraces Recording #78302196956 Logs
DashboardRunWithAgentMcpListTracesReturnsNoTraces_DevLocalhost Recording #78302196956 Logs
DashboardRunWithOtelTracesReturnsNoTraces Recording #78302196956 Logs
DashboardRunWithOtelTracesReturnsNoTraces_DevLocalhost Recording #78302196956 Logs
DeployK8sBasicApiService Recording #78302198240 Logs
DeployK8sWithExternalHelmChart Recording #78302197609 Logs
DeployK8sWithGarnet Recording #78302196995 Logs
DeployK8sWithMongoDB Recording #78302197304 Logs
DeployK8sWithMySql Recording #78302198275 Logs
DeployK8sWithPostgres Recording #78302197289 Logs
DeployK8sWithRabbitMQ Recording #78302196983 Logs
DeployK8sWithRedis Recording #78302197835 Logs
DeployK8sWithSqlServer Recording #78302197260 Logs
DeployK8sWithValkey Recording #78302197525 Logs
DeployTypeScriptAppToKubernetes Recording #78302196699 Logs
DescribeCommandResolvesReplicaNames Recording #78302197444 Logs
DescribeCommandShowsRunningResources Recording #78302197444 Logs
DetachFormatJsonProducesValidJson Recording #78302197880 Logs
DetachFormatJsonProducesValidJsonWhenRestartingExistingInstance Recording #78302197880 Logs
DoPublishAndDeployListStepsWork Recording #78302196903 Logs
DocsCommand_RendersInteractiveMarkdownFromLocalSource Recording #78302197256 Logs
DoctorCommand_DetectsDeprecatedAgentConfig Recording #78302197246 Logs
DoctorCommand_TypeScriptAppHostReportsMissingConfiguredToolchain Recording #78302197377 Logs
DoctorCommand_WithSslCertDir_ShowsTrusted Recording #78302197377 Logs
DoctorCommand_WithoutSslCertDir_ShowsPartiallyTrusted Recording #78302197377 Logs
GatewayWithoutExternalEndpoint_FailsPublishWithGuidance Recording #78302197940 Logs
GeneratedAspireDevScript_StartsWatchMode_WithConfiguredToolchain Recording #78302197605 Logs
GlobalMigration_HandlesCommentsAndTrailingCommas Recording #78302197776 Logs
GlobalMigration_HandlesMalformedLegacyJson Recording #78302197776 Logs
GlobalMigration_PreservesAllValueTypes Recording #78302197776 Logs
GlobalMigration_SkipsWhenNewConfigExists Recording #78302197776 Logs
GlobalSettings_MigratedFromLegacyFormat Recording #78302197776 Logs
IngressWithoutExternalEndpoint_FailsPublishWithGuidance Recording #78302197940 Logs
InitTypeScriptAppHost_AugmentsExistingViteRepoInWorkspaceSubdirectory Recording #78302197605 Logs
InteractiveCSharpInitCreatesExpectedFiles Recording #78302196728 Logs
InvalidAppHostPathWithComments_IsHealedOnRun Recording #78302197094 Logs
JavaScriptHostingApisRunFromTypeScriptAppHost Recording #78302197362 Logs
LatestCliCanStartStableChannelAppHost Recording #78302198336 Logs
LatestCliCanStartStableChannelTypeScriptAppHost Recording #78302198336 Logs
LegacySettingsMigration_AdjustsRelativeAppHostPath Recording #78302198162 Logs
LogsCommandShowsResourceLogs Recording #78302198008 Logs
OtelLogsReturnsStructuredLogsFromStarterApp Recording #78302197191 Logs
OtelLogsReturnsStructuredLogsFromStarterAppIsolated Recording #78302197191 Logs
PsCommandListsRunningAppHost Recording #78302197111 Logs
PsFormatJsonOutputsOnlyJsonToStdout Recording #78302197111 Logs
PublishJavaScriptPatternsGeneratesExpectedDockerComposeArtifacts Recording #78302197987 Logs
PublishWithConfigureEnvFileUpdatesEnvOutput Recording #78302197987 Logs
PublishWithDockerComposeServiceCallbackSucceeds Recording #78302197987 Logs
PublishWithoutOutputPathUsesAppHostDirectoryDefault Recording #78302197987 Logs
ResourceCommand_FailedExecution_DisplaysAppHostLogPathAndLogContainsEntries Recording #78302197840 Logs
ResourceCommand_SetAndDeleteParameterUpdatesDescribeOutput Recording #78302197840 Logs
RestoreGeneratesSdkFiles Recording #78302197981 Logs
RestoreGeneratesSdkFiles_WithConfiguredToolchain Recording #78302197188 Logs
RestoreRefreshesGeneratedSdkAfterAddingIntegration Recording #78302197188 Logs
RestoreSupportsConfigOnlyHelperPackageAndCrossPackageTypes Recording #78302197510 Logs
RunFromParentDirectory_UsesExistingConfigNearAppHost Recording #78302198149 Logs
RunReportsSyntaxErrorsForDotNetAppHost Recording #78302196898 Logs
RunReportsSyntaxErrorsForTypeScriptAppHost Recording #78302196898 Logs
SecretCrudOnDotNetAppHost Recording #78302197005 Logs
SecretCrudOnTypeScriptAppHost Recording #78302197033 Logs
StagingChannel_ConfigureAndVerifySettings_ThenSwitchChannels Recording #78302197691 Logs
StartAndWaitForTypeScriptSqlServerAppHostWithNativeAssets Recording #78302196977 Logs
StartReportsSyntaxErrorsForDotNetAppHost Recording #78302196898 Logs
StartReportsSyntaxErrorsForTypeScriptAppHost Recording #78302196898 Logs
StopAllAppHostsFromAppHostDirectory Recording #78302197081 Logs
StopJavaPolyglotAppHostUsingApphostDirectory Recording #78302197355 Logs
StopNonInteractiveSingleAppHost Recording #78302197081 Logs
StopTypeScriptPolyglotAppHostUsingApphostDirectory Recording #78302197116 Logs
StopWithNoRunningAppHostExitsSuccessfully Recording #78302196188 Logs
UnAwaitedChainsCompileWithAutoResolvePromises Recording #78302197188 Logs
UpdateProjectChannelToStable_CSharpEmptyAppHost_PreservesAspireConfigChannel Recording #78302196555 Logs
UpdateProjectChannelToStable_CSharpSingleFileInit_PreservesAspireConfigChannel Recording #78302196555 Logs
UpdateProjectChannelToStable_TypeScriptSingleFileInit_PreservesAspireConfigChannel Recording #78302196555 Logs
UpdateProjectChannelToStable_TypeScript_PreviewsStablePackagesAndPreservesChannel Recording #78302196555 Logs

📹 Recordings uploaded automatically from CI run #26529541957

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

agentic-workflows area-engineering-systems infrastructure helix infra engineering repo stuff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants