Add automated error message quality linter

[Copilot]

Task: Create Automated Error Message Quality Linter ✅

Successfully created an automated linter to check error messages for compliance with the error message style guide.

✅ All Acceptance Criteria Met

• Linter detects error messages without "Example:" where required
• Linter identifies error messages that don't explain what's expected
• Linter checks for proper YAML formatting in examples
• Linter can be run as make lint-errors
• CI integration added to prevent non-compliant errors from being merged
• Linter provides helpful suggestions for improving detected issues
• Configurable compliance threshold (default 80%)
• Clear, formatted output showing file locations
• Documentation added to CONTRIBUTING.md about error message standards
• Run make lint-errors successfully with baseline threshold set appropriately

📊 Baseline Statistics

• Total error messages analyzed: 786
• Compliant messages: 705 (89%)
• Non-compliant messages: 81 (10%)
• Default threshold: 80%
• Current status: ✅ Exceeds threshold by 9%

🔧 Implementation

Files Created:

1. scripts/lint_error_messages.go - Main linter implementation

   • AST-based Go parser using go/parser and go/ast
   • Detects fmt.Errorf and errors.New calls
   • Validates error messages against quality criteria
   • Generates compliance reports with suggestions

2. scripts/lint_error_messages_test.go - Comprehensive test suite

   • 25+ test cases covering all validation logic
   • Tests for pattern matching, skip logic, and suggestions
   • All tests passing ✅

Files Modified:

1. Makefile - Added lint-errors target and help text
2. .github/workflows/ci.yml - Added linter step to CI pipeline
3. CONTRIBUTING.md - Documented error message standards

Recent Fixes

Commit: Fix linter issues

• Removed unused hasTypeInfo variable
• Added proper error checking for fmt.Sscanf return value
• Improved error handling with warning message for invalid threshold values
• All linter checks now pass ✅

🚀 Usage

# Run the linter with default threshold (80%)
make lint-errors

# Run with custom threshold
go run scripts/lint_error_messages.go 90

# View help
make help | grep lint-errors

🧪 Testing

All tests passing:

• Unit tests: make test-unit ✅
• Linter tests: go test scripts/lint_error_messages_test.go ✅
• Linter validation: make lint ✅
• CI integration: Workflow step added ✅

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[task] Create automated error message quality linter</issue_title>
<issue_description>## Objective
Create an automated linter that checks error messages for compliance with the error message style guide (.github/instructions/error-messages.instructions.md). This should run as part of CI to prevent regression and maintain quality standards.

Context

Part of discussion #4071 - Repository Quality Improvement. Currently at 8% compliance with error message style guide. A linter will help maintain quality as new error messages are added and prevent backsliding on improvements.

Approach

Create a new error message quality linter in scripts/lint_error_messages.go that:

1. Parses all Go files in pkg/workflow/ and pkg/cli/
2. Extracts error messages from fmt.Errorf, errors.New, and similar patterns
3. Checks each error message against quality criteria
4. Reports compliance metrics and specific issues
5. Exits with non-zero code if compliance falls below threshold

Files to Create

• Create: scripts/lint_error_messages.go (new linter)
• Update: Makefile (add lint-errors target)
• Update: .github/workflows/ci.yml or relevant CI workflow (add linter step)

Implementation Details

Linter Functionality

The linter should:

1. Parse Go files using go/parser and go/ast
2. Detect error patterns:
   • fmt.Errorf(...) calls
   • errors.New(...) calls
   • Function returns with error type
3. Check error message quality:
   • Contains "Example:" when appropriate (validation, configuration errors)
   • Includes "Expected" or "Valid" for format/enum errors
   • Shows received value/type for type mismatches (%T, %v)
   • Uses proper YAML formatting in examples
4. Report metrics:
   • Total error messages analyzed
   • Compliant count and percentage
   • Non-compliant messages with (redacted) location
   • Suggestions for improvement
5. Configurable threshold: Exit code 1 if compliance < 80% (configurable)

Makefile Integration

lint-errors:
	`@echo` "Running error message quality linter..."
	`@go` run scripts/lint_error_messages.go

CI Integration

Add to GitHub Actions workflow (in test/lint stage):

- name: Lint error messages
  run: make lint-errors

Example Output

🔍 Error Message Quality Linter

Analyzing error messages in pkg/workflow/ and pkg/cli/...

✓ pkg/workflow/time_delta.go: 5/5 compliant (100%)
✗ pkg/workflow/engine_validation.go: 3/12 compliant (25%)
  - Line 45: Missing example for engine validation
    Suggestion: Add "Example: engine: copilot"
  - Line 78: No valid options listed for mode
    Suggestion: Add "Valid modes: local, remote"

📊 Summary:
  Total error messages: 237
  Compliant: 189 (80%)
  Non-compliant: 48 (20%)
  
✅ Meets quality threshold (80%)

Acceptance Criteria

• Linter detects error messages without "Example:" where required
• Linter identifies error messages that don't explain what's expected
• Linter checks for proper YAML formatting in examples
• Linter can be run as make lint-errors
• CI integration added to prevent non-compliant errors from being merged
• Linter provides helpful suggestions for improving detected issues
• Configurable compliance threshold (default 80%)
• Clear, formatted output showing (redacted) locations
• Documentation added to CONTRIBUTING.md about error message standards
• Run make lint-errors successfully with baseline threshold set appropriately

AI generated by Plan Command for discussion #4071</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes [task] Create automated error message quality linter #4076

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[pelikhan]

@copilot lint

[Copilot]

Pull Request Overview

This PR implements an automated error message quality linter to enforce the error message style guide template: [what's wrong]. [what's expected]. [example]. The linter uses AST-based parsing to extract error messages from Go code, validates them against quality criteria, and reports compliance metrics. Currently, 89% (705/786) of error messages comply with the standard.

Key Changes

• AST-based linter with configurable compliance threshold (default 80%)
• CI integration to prevent non-compliant error messages from being merged
• Comprehensive documentation in CONTRIBUTING.md with error message guidelines and examples

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
scripts/lint_error_messages.go	New linter implementation with AST parsing, regex-based validation, and suggestion generation
scripts/lint_error_messages_test.go	Comprehensive test suite covering error quality detection, skip logic, suggestions, and pattern matching
Makefile	Added lint-errors target for running the linter
.github/workflows/ci.yml	Integrated linter into CI lint job
CONTRIBUTING.md	Added error message style guide section with examples and best practices

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The hasTypeInfo regex pattern is defined but never used in the code. Consider removing this unused variable or documenting its intended future use.

Suggested change

hasTypeInfo = regexp.MustCompile(`%[TvdsFfgGeExXbcqpoUq]`)

[Copilot]

@copilot lint

Fixed linter issues in commit df79f9a:

Issues resolved:

• Removed unused hasTypeInfo variable
• Added proper error checking for fmt.Sscanf return value with warning message for invalid threshold values

All linter checks now pass ✅