-
Notifications
You must be signed in to change notification settings - Fork 330
[cli-tools-test] compile: unknown tool names in frontmatter are silently ignored without warning #24835
Description
Problem Description
When a workflow markdown file contains an unknown/invalid tool name in the tools: frontmatter section, the compiler silently ignores it without emitting any validation error or warning. The user receives no indication that their tool configuration was not applied.
Steps to Reproduce
- Create a workflow markdown file with an unknown tool in the
tools:section:
---
engine: copilot
on:
schedule:
- cron: "0 0 * * *"
tools:
nonexistent-tool-xyz:
version: "1.0"
network:
allowed:
- github.com
---
This is a test workflow.
```
2. Compile the workflow using `gh aw compile` or the MCP `compile` tool.
### Expected Behavior
The compiler should emit a **validation warning** (or error in strict mode) indicating that `nonexistent-tool-xyz` is not a recognized tool and was ignored. Something like:
```
WARNING: tools.nonexistent-tool-xyz: unknown tool name, will be ignoredActual Behavior
The workflow compiles successfully with no errors and no warnings. The unknown tool entry is completely silent — it does not appear in the generated .lock.yml file and there is no indication to the user that their configuration was not applied.
Environment
- Repository: github/gh-aw
- Run ID: §24019945745
- Date: 2026-04-06
- Test: Daily CLI Tools Exploratory Tester
Impact
- Severity: Medium
- Frequency: Always (reproducible with any unknown tool name)
- Workaround: None — users have no way to know their tool configuration is being ignored unless they manually inspect the generated lock file
This is especially problematic because:
- Users may spend time debugging why a tool isn't working without realizing the name is wrong
- Typos in built-in tool names (e.g.,
playwrigjhtinstead ofplaywright) would also be silently ignored - There is no feedback loop to help users discover valid tool names
Additional Context
Tested with nonexistent-tool-xyz as the tool name. The generated lock file contained no trace of the tool configuration. Strict mode (strict: true) does not catch this case either — the behavior is the same regardless of strict mode setting.
References:
Generated by Daily CLI Tools Exploratory Tester · ● 1.9M · ◷
- expires on Apr 13, 2026, 5:34 AM UTC