-
Notifications
You must be signed in to change notification settings - Fork 28
Add inline examples to schema validation errors #4080
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Enhance schema validation error messages to provide field-specific examples for common workflow fields. When schema validation fails, users now receive actionable inline examples showing correct YAML syntax. - Add enhanceSchemaValidationError() to parse ValidationError and add examples - Add extractFieldPath() to extract field name from InstanceLocation - Add getFieldExample() with map of 22 common workflow field examples - Add generic type-based examples for unknown fields (string, boolean, object, array, integer) - Add comprehensive unit tests for field extraction, example generation, and error enhancement - Add integration test for schema validation with examples - Add example quality test to verify all field examples contain required parts Example improvements: - timeout-minutes: "Example: timeout-minutes: 10" - engine: "Valid engines are: copilot, claude, codex, custom. Example: engine: copilot" - permissions: "Example: permissions:\n contents: read\n issues: write" Fixes #4071 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR enhances schema validation error messages to provide inline, field-specific examples rather than requiring users to reference external documentation. When a GitHub Actions schema validation fails, users now receive helpful examples showing correct YAML syntax for the field in question.
Key changes:
- Added intelligent error enhancement to provide context-aware examples based on the failing field
- Implemented a comprehensive field example map covering 22 common workflow fields with fallback to generic type-based examples
- Added thorough test coverage including unit tests and integration tests for example quality
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| pkg/workflow/schema_validation.go | Enhanced with enhanceSchemaValidationError(), extractFieldPath(), and getFieldExample() functions to wrap validation errors with examples |
| pkg/workflow/schema_validation_test.go | Added comprehensive unit tests for field extraction, example generation, and error enhancement logic |
| pkg/workflow/schema_validation_integration_test.go | Added integration tests to verify workflow compilation and validate example quality for common fields |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "permissions": "Example: permissions:\\n contents: read\\n issues: write", | ||
| "on": "Example: on: push or on:\\n issues:\\n types: [opened]", | ||
| "runs-on": "Example: runs-on: ubuntu-latest", | ||
| "concurrency": "Example: concurrency: production or concurrency:\\n group: ${{ github.workflow }}\\n cancel-in-progress: true", | ||
| "env": "Example: env:\\n NODE_ENV: production", | ||
| "tools": "Example: tools:\\n github:\\n allowed: [list_issues]", | ||
| "steps": "Example: steps:\\n - name: Checkout\\n uses: actions/checkout@v4", | ||
| "jobs": "Example: jobs:\\n build:\\n runs-on: ubuntu-latest\\n steps:\\n - run: echo 'hello'", | ||
| "strategy": "Example: strategy:\\n matrix:\\n os: [ubuntu-latest, windows-latest]", | ||
| "container": "Example: container: node:20 or container:\\n image: node:20\\n options: --user root", | ||
| "services": "Example: services:\\n postgres:\\n image: postgres:15\\n env:\\n POSTGRES_PASSWORD: postgres", | ||
| "defaults": "Example: defaults:\\n run:\\n shell: bash", | ||
| "name": "Example: name: \"Build and Test\"", | ||
| "if": "Example: if: github.event_name == 'push'", | ||
| "environment": "Example: environment: production or environment:\\n name: production\\n url: https://example.com", | ||
| "outputs": "Example: outputs:\\n build-id: ${{ steps.build.outputs.id }}", | ||
| "needs": "Example: needs: build or needs: [build, test]", | ||
| "uses": "Example: uses: ./.github/workflows/reusable.yml", | ||
| "with": "Example: with:\\n node-version: '20'", | ||
| "secrets": "Example: secrets:\\n token: ${{ secrets.GITHUB_TOKEN }}", |
Copilot
AI
Nov 15, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The newlines in the example strings are double-escaped (\\\\n), which will render as literal \\n in error messages instead of actual line breaks. They should use single backslash (\\n) or use raw string literals with actual newlines.
| return fmt.Sprintf("Example: %s: true", fieldPath) | ||
| } | ||
| if strings.Contains(errorMsg, "object") { | ||
| return fmt.Sprintf("Example: %s:\\n key: value", fieldPath) |
Copilot
AI
Nov 15, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The newline is double-escaped (\\\\n), which will render as literal \\n in the error message instead of an actual line break. It should use single backslash (\\n).
| return fmt.Sprintf("Example: %s:\\n key: value", fieldPath) | |
| return fmt.Sprintf("Example: %s:\n key: value", fieldPath) |
Schema validation errors now include field-specific inline examples following the error message pattern: [what's wrong] + [what's expected] + [example].
Changes
Enhanced error messages (
pkg/workflow/schema_validation.go):enhanceSchemaValidationError()to wrap ValidationError with field-specific examplesextractFieldPath()to extract field name from InstanceLocationgetFieldExample()with 22 common workflow field examples + generic type fallbacksTest coverage (2 new test files):
Example
Before:
After:
For enum fields:
For complex fields:
Generic type-based examples (string, boolean, object, array, integer) are provided for fields not in the predefined map.
Fixes #4071
Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.