Fix failing tests after permissions validation changes#2831
Merged
Conversation
- Fix AI reaction tests: count jobs by name instead of runs-on - Fix permissions extraction test: include all permissions in output - Fix invalid YAML tests: correct line numbers for errors - Fix custom permissions test: remove pull-requests from frontmatter - Fix PR checkout tests: update permissions and expectations Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Added issues and pull-requests read permissions to integration test workflows to satisfy new permission validation requirements. Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Fix existing test cases for stability
Fix failing tests after permissions validation changes
Oct 30, 2025
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR fixes test failures caused by recent changes that introduced stricter permission validation for GitHub toolsets. The changes update test expectations and add required permissions to ensure tests pass with the new validation requirements.
Key changes:
- Updated job counting logic to use job names instead of
runs-onvalues - Corrected YAML error line number expectations to match current parser output
- Added required
contents: readandpull-requests: readpermissions to test workflows
Reviewed Changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/workflow/compiler_test.go | Updated job counting to check job names; corrected YAML error line numbers; removed pull-requests: read from custom permissions test |
| pkg/workflow/permissions_import_test.go | Fixed test to expect pull-requests: read in extracted permissions output |
| pkg/workflow/pr_checkout_test.go | Added required permissions and updated expectations from false to true |
| pkg/workflow/pr_ready_for_review_checkout_test.go | Added required permissions and updated expectations from false to true |
| pkg/cli/compile_integration_test.go | Added issues: read and pull-requests: read to satisfy validation requirements |
| pkg/workflow/schemas/github-workflow.json | Formatting change: condensed multi-line arrays to single lines |
| docs/src/content/docs/status.mdx | Added two new workflow entries to status table |
| docs/src/content/docs/reference/frontmatter-full.md | Documentation updates for engine configuration and safe-outputs |
| .github/workflows/*.lock.yml | Updated actions/setup-node action versions |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
|
Agentic Changeset Generator triggered by this pull request. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recent changes introduced stricter permission validation for GitHub toolsets, breaking tests that predated this requirement.
Changes
Job counting logic (compiler_test.go)
runs-on: ubuntu-latestto checking job names (pre_activation,activation,agent)pre_activationjob now usesubuntu-slim, breaking the previous countYAML error line numbers (compiler_test.go)
unclosed_bracket_in_array: 9 → 10invalid_mapping_context: 6 → 7unclosed_quote: 8 → 9duplicate_keys: 5 → 7Permission expectations (permissions_import_test.go, compiler_test.go, pr_checkout_test.go, pr_ready_for_review_checkout_test.go)
TestExtractPermissionsFromContentto expect all permissions in output (was expectingpull-requeststo be filtered)pull-requests: readfromTestCustomPermissionsOverrideDefaultsfrontmatter to match test intentcontents: readandpull-requests: readto PR checkout testsfalsetotruesince workflows now have required permissionsIntegration tests (compile_integration_test.go)
issues: readandpull-requests: readto test workflows to satisfy new validation requirementsOriginal prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.