-
Notifications
You must be signed in to change notification settings - Fork 28
Enhance engine validation error messages with examples and valid options #4077
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- Improved error messages in engine_validation.go following style guide - Added count to multiple engine specifications error for clarity - Enhanced parse error with proper Example (string) and (object) labels - Made invalid configuration error more specific about missing 'id' field - Used escaped newlines (\\n) for proper YAML formatting in examples - Added comprehensive test coverage in engine_validation_test.go - All tests pass successfully Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR enhances engine validation error messages to follow the established style guide template: [what's wrong] + [what's expected] + [example]. The improvements make error messages more actionable by adding counts, format examples, and valid options.
Key Changes:
- Added engine specification count to multiple engines error for better clarity
- Enhanced parse failure errors with labeled format examples (string and object)
- Improved invalid configuration errors by specifying root cause (missing/invalid 'id' field)
- Added comprehensive test coverage to validate error message quality
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| pkg/workflow/engine_validation.go | Enhanced three error messages with counts, examples, and valid options following style guide |
| pkg/workflow/engine_validation_test.go | Added two new test functions with subtests to validate error message quality and completeness |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| } | ||
|
|
||
| // Error should include an example | ||
| if !strings.Contains(errorMsg, "Example:") && !strings.Contains(errorMsg, "engine: copilot") { |
Copilot
AI
Nov 15, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test assertion uses incorrect logical operator (AND &&). The condition should be !strings.Contains(errorMsg, "Example:") || !strings.Contains(errorMsg, "engine: copilot") instead, because the test should fail if the error message is missing EITHER "Example:" OR the example content. With AND, the test only fails if BOTH are missing, which would allow an error message with just "engine: copilot" but no "Example:" prefix to pass.
| if !strings.Contains(errorMsg, "Example:") && !strings.Contains(errorMsg, "engine: copilot") { | |
| if !strings.Contains(errorMsg, "Example:") || !strings.Contains(errorMsg, "engine: copilot") { |
| } | ||
|
|
||
| // Error should include example | ||
| if !strings.Contains(errorMsg, "Example:") && !strings.Contains(errorMsg, "engine: copilot") { |
Copilot
AI
Nov 15, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test assertion uses incorrect logical operator (AND &&). The condition should be !strings.Contains(errorMsg, "Example:") || !strings.Contains(errorMsg, "engine: copilot") instead, because the test should fail if the error message is missing EITHER "Example:" OR the example content. With AND, the test only fails if BOTH are missing, which would allow an error message with just "engine: copilot" but no "Example:" prefix to pass.
| if !strings.Contains(errorMsg, "Example:") && !strings.Contains(errorMsg, "engine: copilot") { | |
| if !strings.Contains(errorMsg, "Example:") || !strings.Contains(errorMsg, "engine: copilot") { |
Engine validation errors lacked actionable guidance. Users encountered messages like "invalid engine configuration in included file" without examples or valid options listed.
Changes
Error Message Enhancements
Multiple engine specifications (L93)
(%d engine specifications detected)for clarity on duplication extentParse failure (L104)
Example (string):and(object):modelfield\\nfor proper YAML displayInvalid configuration (L119)
missing or invalid 'id' fieldExpected string or object with 'id' fieldTest Coverage
Added
TestValidateEngineErrorMessageQualityandTestValidateSingleEngineSpecificationErrorMessageQualitywith subtests validating:Example
Before:
After:
Follows error message style guide template: [what's wrong] + [what's expected] + [example].
Original prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.