Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Nov 15, 2025

Strict mode validation errors lacked guidance on why restrictions exist and how to achieve goals safely. Users encountering errors had no clear path to resolution.

Changes

Error message enhancements in pkg/workflow/strict_mode_validation.go:

  • Write permissions: Links to safe-outputs alternatives (create-issue, create-pull-request, add-comment, update-issue)
  • Network configuration: Explains prevention of unrestricted access, suggests network.allowed or ecosystem identifiers
  • Wildcard domains: Suggests explicit domains or ecosystem IDs (python, node, containers) instead of *
  • MCP network config: Explains security requirement for containerized servers

Test updates:

  • Modified assertions in strict_mode_validation_test.go to verify hint content presence
  • Updated validation_strict_mcp_network_test.go to use partial message matching

Example

Before:

✗ strict mode: write permission 'contents: write' is not allowed

After:

✗ strict mode: write permission 'contents: write' is not allowed for security reasons. 
  Use 'safe-outputs.create-issue', 'safe-outputs.create-pull-request', 
  'safe-outputs.add-comment', or 'safe-outputs.update-issue' to perform write 
  operations safely. See: https://githubnext.github.io/gh-aw/reference/safe-outputs/

All error messages now include security rationale (WHY) and actionable alternatives (HOW) with documentation links.

Original prompt

This section details on the original issue you should resolve

<issue_title>[task] Add Actionable Hints to Strict Mode Validation Errors</issue_title>
<issue_description>## Objective
Enhance strict mode validation error messages to include actionable hints about alternative approaches, explaining WHY restrictions exist and HOW to achieve goals safely.

Context

Part of Discussion #3956 - Workflow Validation and Error Feedback Quality improvements.

When strict mode rejects an operation, users need to understand the security rationale and learn safe alternatives (like using safe-outputs).

Implementation Approach

Update Error Messages in pkg/workflow/strict_mode_validation.go

Transform all strict mode errors to include:

  1. Security rationale (WHY)
  2. Safe alternative approaches (HOW)
  3. Documentation links

Example transformations:

1. Write Permissions Error

// Before:
fmt.Errorf("strict mode: write permission '%s: write' is not allowed", scope)

// After:
fmt.Errorf("strict mode: write permission '%s: write' is not allowed for security reasons. Use 'safe-outputs.create-issue' or 'safe-outputs.create-pull-request' to perform write operations safely. See: (redacted)", scope)

2. Network Configuration Error

// Before:
fmt.Errorf("strict mode: 'network' configuration is required")

// After:
fmt.Errorf("strict mode: 'network' configuration is required to prevent unrestricted network access. Add 'network: { allowed: [...] }' or 'network: defaults' to your frontmatter. See: (redacted)")

3. Wildcard Error

// Before:
fmt.Errorf("strict mode: wildcard '*' is not allowed in network.allowed domains")

// After:  
fmt.Errorf("strict mode: wildcard '*' is not allowed in network.allowed domains to prevent unrestricted internet access. Specify explicit domains or use ecosystem identifiers like 'python', 'node', 'containers'. See: (redacted)#ecosystem-identifiers")

Use Enhanced Error Formatting

  • Leverage console.CompilerError struct with Hint field for better formatting
  • Ensure all errors use console.FormatErrorMessage

Files to Modify

  • Update: pkg/workflow/strict_mode_validation.go
  • Update: pkg/workflow/strict_mode_validation_test.go (add hint verification tests)

Acceptance Criteria

  • All 4+ strict mode error messages include actionable hints
  • Error messages explain security rationale (WHY)
  • Error messages suggest safe-outputs alternatives where applicable (HOW)
  • Error messages use console.FormatErrorMessage with hint field
  • Documentation links are valid and helpful
  • Unit tests verify hint content is present in error output
  • All tests pass (make test-unit)

Priority

High - Quick win that significantly improves security guidance and user experience
Related to #3956

AI generated by Plan Command for discussion #3956</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

- Enhanced all 4 strict mode error messages with:
  - Security rationale (WHY restrictions exist)
  - Safe alternative approaches (HOW to achieve goals safely)
  - Documentation links to relevant reference pages
- Updated test assertions to verify hint content is present
- All tests passing with new enhanced error messages

Error messages now include:
1. Write permissions: Suggests safe-outputs alternatives
2. Network configuration: Explains network.allowed and ecosystem identifiers
3. Wildcard domains: Suggests explicit domains and ecosystem IDs
4. MCP network config: Explains network configuration for containers

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add actionable hints to strict mode validation errors Add actionable hints to strict mode validation errors Nov 15, 2025
Copilot AI requested a review from pelikhan November 15, 2025 05:19
Copilot finished work on behalf of pelikhan November 15, 2025 05:19
@pelikhan pelikhan marked this pull request as ready for review November 15, 2025 06:02
Copilot AI review requested due to automatic review settings November 15, 2025 06:02
@github-actions
Copy link
Contributor

Agentic Changeset Generator triggered by this pull request.

Copilot finished reviewing on behalf of pelikhan November 15, 2025 06:04
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request enhances strict mode validation error messages to include security rationale and actionable guidance for users, making it easier to understand why restrictions exist and how to achieve goals safely.

Key improvements:

  • All strict mode error messages now explain the security reason for the restriction
  • Error messages include specific alternatives (safe-outputs, network configuration examples)
  • Documentation links are provided for further guidance
  • Tests updated to use partial string matching to accommodate enhanced messages

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
pkg/workflow/strict_mode_validation.go Enhanced all 4 strict mode validation error messages with security rationale, actionable alternatives, and documentation links
pkg/workflow/strict_mode_validation_test.go Updated test assertions to verify error messages contain expected security guidance using partial string matching
pkg/workflow/validation_strict_mcp_network_test.go Changed test assertion from exact match to partial match using strings.Contains to accommodate enhanced error message

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@pelikhan pelikhan merged commit d47c8e9 into main Nov 15, 2025
4 checks passed
@pelikhan pelikhan deleted the copilot/add-actionable-hints-to-errors branch November 15, 2025 06:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[task] Add Actionable Hints to Strict Mode Validation Errors

2 participants