feat(ai-step): new small composite action for structured AI output - #131
Merged
Conversation
Callers want an ai-in / JSON-out primitive: pass a prompt + input + JSON Schema, get back schema-conforming JSON on steps.*.outputs.result that downstream steps can branch on via fromJSON(). ai-pr-review owns the PR-review preset (comments, sticky summary, provenance) — the wrong shape for "classify this diff," "pick a reviewer," or "extract fields from a changelog." Generalizing ai-pr-review would bloat its input surface and couple unrelated concerns. ai-step is the small building block: wraps claude-code-action with --json-schema and codex-action with output-schema, unifies the outputs, never hard-fails. The caller decides how to react to empty or unexpected JSON — structured output is the contract, caller owns the response policy. Closes DEVOPS-834
GitHub Actions parses any ${{ ... }} it sees inside action.yml —
including prose in a description field — as an expression. The
placeholder example in the 'input' description triggered:
Unrecognized named-value: 'steps'
on every caller, breaking composite-action loading before it ran.
Rewrite the description in plain prose; the usage examples in the
README keep the templated form.
claude-code-action tries to exchange an OIDC token for a GitHub App token on startup unless github_token is explicitly provided. That fails with 'Could not fetch an OIDC token' in any caller workflow that doesn't set id-token: write — a surprising requirement for what is meant to be a pure text-to-JSON primitive with no PR interaction. Passing github.token (always available to composite actions) makes id-token: write unnecessary for ai-step callers.
The claude-code-action wrapper was the wrong dependency for a generic text-to-JSON primitive: it ships a full PR-review pipeline (bun install, sandboxing, review-mode cleanup) that we don't need, which cost ~90s cold-start per call and exposed bugs unrelated to schema binding (tsconfig directory-mismatch; --json-schema hangs when combined with default output-format; OIDC dance even for pure API calls). Both providers ship first-class structured-output support on their chat APIs — Anthropic via output_config.format.schema on the Messages API, OpenAI via response_format.json_schema.schema on Chat Completions. Using the official python SDKs gives us native schema binding, proper error types, and a ~3-5s end-to-end call in place of ~90s, with none of the wrapper-specific failure modes. Losing tool-use/MCP support in v1 is intentional; callers who need it can reach for claude-code-action directly.
Anthropic structured-output (and OpenAI strict mode) reject schemas where an object type doesn't explicitly set additionalProperties=false. First smoke call failed with 'For object type, additionalProperties must be explicitly set to false' — annoying for every caller to repeat on every nested object. Walk the parsed schema once before sending and set the flag where missing, leaving explicit values alone.
…hema fields Previous docs described a claude-code-action / codex-action wrapper and listed tool/mcp-config inputs we removed when we moved to direct SDK calls. Also relax the in-repo smoke schema to drop numeric min/max so the same schema works on Anthropic strict mode.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.github/actions/ai-stepcomposite: prompt + input + JSON Schema → schema-conforming JSON onsteps.<id>.outputs.result. Caller parses viafromJSON(...)and branches on typed fields.output_config.format.schema) and OpenAI (Chat Completions +response_format.json_schema.schema). No wrappers, no bun, no PR-review machinery.additionalProperties: falseon object nodes (both providers require it in strict mode).conclusion=failedwith upstream body in the CI log, caller decides.Why this, not generalize
ai-pr-reviewai-pr-reviewis PR-shaped — owns checkout, commenting, sticky summaries, provenance footer.ai-stepis the primitive for any classify/extract/route flow that doesn't need PR context. Generalizing the review action would have coupled unrelated concerns.An earlier draft wrapped
claude-code-action/codex-action. Live testing showed ~90s cold-start plus a bun directory-mismatch hang unrelated to schema binding — both rooted in the wrappers' PR-review machinery. Direct SDK calls run end-to-end in ~15s and sidestep that entirely.Live-tested
Both providers ran the same schema + prompt in the
devops-scratchpadsmoke workflow:Test plan
make test-ai-step— 14/14 bats tests pass forresolve-config.sh(happy path per provider × effort, invalid inputs, missing envs, empty schema).loft-sh/devops-scratchpadrun 24732067259: schema conformance verified,fromJSON()branching exercised.loft-sh/devops-scratchpadrun 24732244920: same schema, same contract.docs/workflows/ai-step.mdfor the design rationale (why direct-SDK won over the wrapper approach)..github/actions/ai-step/README.mdagainst your own schema before cutting theai-step/v1tag.Closes DEVOPS-834