fix: correct AI agent skills and add CLI example validation - #58
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughChangesSkill validation and documentation
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant SkillFiles
participant skills_test
participant a6
SkillFiles->>skills_test: Provide shell and YAML examples
skills_test->>skills_test: Normalize and extract a6 invocations
skills_test->>a6: Invoke --help
a6-->>skills_test: Return commands and flags
skills_test-->>SkillFiles: Validate command paths and flags
🚥 Pre-merge checks | ✅ 5 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eb8b5a8d12
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Makefile`:
- Around line 88-89: Declare the aggregate check target as phony by adding check
to the existing .PHONY declaration, while preserving its current fmt, vet, lint,
test, validate-skills, and test-skills dependencies.
In `@skills/a6-persona-operator/SKILL.md`:
- Line 120: Remove the duplicate `a6 --context staging route list` command near
the existing `a6 --context staging route list --output table` check, or replace
it with the intended distinct verification so staging routes are not listed
twice.
In `@skills/a6-plugin-jwt-auth/SKILL.md`:
- Around line 152-157: Add a separate RS256 credential example for Bob instead
of reusing credential.yaml: define bob-rs256-credential.yaml with Bob’s key,
algorithm set to RS256, and the public key, then update the a6 credential create
command to reference that filename.
In `@test/skills/skills_test.go`:
- Around line 150-153: Update the subcommand validation around
availableCommands(help) so that when help lists subcommands, any next non-flag
token not present in subcommands causes the test to fail instead of being
treated as a positional argument. Preserve the existing handling for flags and
valid nested commands, and ensure unknown cases such as “a6 route typo” are
rejected.
- Around line 17-178: Replace inferred local declarations in locateRepoRoot,
TestMain, TestSkillShellExamplesUseSupportedA6CommandsAndFlags,
availableCommands, and related helpers with explicit Go types, including dir,
err, root, commands, and other := variables. Preserve the existing control flow
and behavior while ensuring every local variable declaration states its type
explicitly.
- Line 15: Remove the package-level mutable variable a6Binary and create the
binary path through a test helper or factory. Update the affected tests and
validation functions to receive that path explicitly rather than reading global
state, using the existing factory pattern where available.
- Around line 35-41: Update TestMain’s error paths for locateRepoRoot and
os.MkdirTemp to report each err to stderr before calling os.Exit(1), including
context identifying the failed operation.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 7d92d73f-6328-4b74-8a7b-468408358a17
📒 Files selected for processing (12)
.github/workflows/ci.ymlMakefiledocs/skills.mdskills/a6-persona-developer/SKILL.mdskills/a6-persona-operator/SKILL.mdskills/a6-plugin-basic-auth/SKILL.mdskills/a6-plugin-consumer-restriction/SKILL.mdskills/a6-plugin-hmac-auth/SKILL.mdskills/a6-plugin-jwt-auth/SKILL.mdskills/a6-plugin-key-auth/SKILL.mdskills/a6-recipe-multi-tenant/SKILL.mdtest/skills/skills_test.go
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e5bdfd6aa3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/a6-plugin-jwt-auth/SKILL.md`:
- Around line 161-162: Update the instructions surrounding
bob-rs256-credential.yaml to tell readers to paste only the PEM body from
public.pem, excluding the BEGIN PUBLIC KEY and END PUBLIC KEY delimiter lines
already present in the YAML.
In `@test/skills/skills_test.go`:
- Around line 221-228: Update resolveCommand’s field-scanning loop to consume
supported global flags such as --server together with their values before
continuing nested-command validation, so later subcommands like creat are still
rejected. For unsupported interspersed options, fail closed rather than
resolving only the preceding command, and add a regression case covering the
--server value before the invalid nested command.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c46d53e2-e192-4737-9681-2c5b08a457f2
📒 Files selected for processing (6)
Makefileskills/a6-persona-developer/SKILL.mdskills/a6-persona-operator/SKILL.mdskills/a6-plugin-jwt-auth/SKILL.mdskills/a6-recipe-multi-tenant/SKILL.mdtest/skills/skills_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
- Makefile
- skills/a6-persona-operator/SKILL.md
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2b272806a5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1d710c21fb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
This PR updates the a6 Skills catalog to align examples with the current CLI surface (command names, flags, and credential subresources), and adds automated validation to prevent stale shell examples from slipping through.
Changes:
- Updated multiple
skills/*/SKILL.mdfiles to use current commands/flags and thecredentialsubresource workflow. - Added a Go test that builds the current
a6CLI and validates skill shell examples against the Cobra help output (commands + long flags). - Wired the new validation into
make checkand CI.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
test/skills/skills_test.go |
New test that parses skill shell blocks, resolves command paths via a6 --help, and validates long flags per-command + global flags. |
skills/a6-recipe-multi-tenant/SKILL.md |
Moves key-auth data from consumer inline plugins to a6 credential create examples. |
skills/a6-plugin-key-auth/SKILL.md |
Replaces raw Admin API curl credential example with a6 credential create. |
skills/a6-plugin-jwt-auth/SKILL.md |
Migrates credential examples to a6 credential create and updates RS256 walkthrough to include consumer creation + YAML credential file. |
skills/a6-plugin-hmac-auth/SKILL.md |
Replaces curl credential example with a6 credential create. |
skills/a6-plugin-basic-auth/SKILL.md |
Replaces curl credential example with a6 credential create. |
skills/a6-plugin-consumer-restriction/SKILL.md |
Updates command name to a6 consumer-group create. |
skills/a6-persona-operator/SKILL.md |
Updates stale commands/flags (e.g., health, debug trace args/flags, logs usage) to current CLI. |
skills/a6-persona-developer/SKILL.md |
Updates examples to current CLI (route list, config dump, debug trace args/flags) and migrates credentials. |
Makefile |
Adds test-skills target and includes it (and validate-skills) in check. |
docs/skills.md |
Documents per-skill installation + approval-first usage guidance; documents new validation (make test-skills). |
.github/workflows/ci.yml |
Adds a validate-skills job step to run the new make test-skills validation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/skills/skills_test.go`:
- Around line 305-309: Update command resolution around the root flag validation
to supply all supported root flags, including the short -o alias for --output,
instead of deriving rootFlags only from long flags via availableFlags and
longFlagPattern. Keep long-flag validation behavior separate while ensuring a6
route -o yaml get ... is accepted.
- Around line 161-182: Update the YAML run-block parsing around the
folded-scalar handling in the skills test so blocks beginning with `>` replace
nonblank line breaks with spaces before being passed to `joinedShellLines`,
while preserving blank-line behavior and literal `|` blocks. Add a regression
case where a command’s continuation line contains an unsupported long flag,
ensuring validation sees it as part of the same command.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 4ae5b134-863d-477c-9a2f-e57fb3062179
📒 Files selected for processing (2)
skills/a6-plugin-jwt-auth/SKILL.mdtest/skills/skills_test.go
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6d97ddeb35
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/a6-recipe-multi-tenant/SKILL.md`:
- Around line 13-17: Add every distinct command used by the skill to the
a6_commands inventory, including consumer create, consumer-group create, route
create, upstream die, route update, config diff, config sync, credential create,
consumer-group list, and consumer get. Preserve the existing entries and avoid
duplicates.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: e68575cc-1eac-43f6-87ae-4b86c9f43534
📒 Files selected for processing (8)
skills/a6-persona-developer/SKILL.mdskills/a6-persona-operator/SKILL.mdskills/a6-plugin-basic-auth/SKILL.mdskills/a6-plugin-hmac-auth/SKILL.mdskills/a6-plugin-jwt-auth/SKILL.mdskills/a6-plugin-key-auth/SKILL.mdskills/a6-recipe-multi-tenant/SKILL.mdtest/skills/skills_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
- skills/a6-persona-operator/SKILL.md
kayx23
changed the title
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/a6-recipe-multi-tenant/SKILL.md`:
- Around line 42-46: Update the route configuration and request flow described
in the recipe so traffic-split cannot trust a client-supplied http_x_tenant_id;
derive or overwrite it from the authenticated Consumer/Consumer Group before
routing. Preserve tenant-specific upstream selection for valid identities, and
add a cross-tenant integration test proving header spoofing cannot select
another tenant’s upstream.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 35472d5b-00aa-44ce-964a-6f30fe0c7794
📒 Files selected for processing (1)
skills/a6-recipe-multi-tenant/SKILL.md
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2de8c6d48f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bb1ff2c2b0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| func TestSkillCommandsUseSupportedA6CommandsAndFlags(t *testing.T) { | ||
| var shellFencePattern *regexp.Regexp = regexp.MustCompile("(?s)```(?:bash|sh|shell)\\s*\\n(.*?)```") | ||
| var yamlFencePattern *regexp.Regexp = regexp.MustCompile("(?s)```(?:yaml|yml)\\s*\\n(.*?)```") | ||
| var longFlagPattern *regexp.Regexp = regexp.MustCompile(`--[a-z][a-z0-9-]*`) |
There was a problem hiding this comment.
When a flag typo starts with an uppercase letter, such as a6 route list --Output json, this pattern does not match it. Because resolveCommand stops after resolving the leaf command, the unsupported-flag loop never sees --Output, so CI accepts an example that Cobra will reject at runtime. Match every syntactically possible long-option token and use validFlags to determine whether it is supported.
AGENTS.md reference: AGENTS.md:L179-L179
Useful? React with 👍 / 👎.
Summary
This PR covers two parts: correcting the skill documentation and adding automated validation tests with CI integration.
make checkWhy
The Skills Hub audit found that the existing frontmatter validation could pass while skill bodies still referenced removed commands, flags, and the old inline credential model. Agents may execute these examples directly, so command examples need validation against the released CLI surface.
APISIX compatibility
The affected authentication, developer, and multi-tenant skills now declare
apisix_version: ">=3.11.0"because their documented workflows usea6 credential createand the APISIX Consumer Credential API introduced in APISIX 3.11. The authentication plugins themselves predate 3.11; this minimum applies to following these skill workflows as written. Supporting APISIX 3.0–3.10 would require a separate legacy workflow that stores authentication plugin configuration inline on Consumers.Runtime and behavioral impact
Together, this PR and the companion a7 PR #73 do not change a6, a7, API7 Gateway, or APISIX runtime behavior. Across the two PRs, the changes are limited to:
make checkin a6End-user software behavior stays the same. AI agents following these skills may behave differently because they now receive correct commands; that is the intended effect.
Validation
make validate-skillsmake test-skillsmake checkDistribution after merge
No new a6 release is required. Unpinned skill installations use the
mainbranch, so installs after merge receive the corrections; existing installed copies require an update or reinstall. Existing release tags and the skills'version: "1.0.0"metadata remain unchanged because this PR does not introduce a separate skill-release policy.Summary by CodeRabbit
Documentation
Tests
Chores