Investigate Copilot small model-alias failure in smoke workflow

actions/setup/js/models.json

@@ -262,7 +262,25 @@
           "cost": {
             "input": "2.5e-07",
             "output": "2e-06",
-            "cache_read": "2.5000000000000002e-08"
+            "cache_read": "2.5e-08"
+          },
+          "provider_type": "openai",
+          "wire_api": "responses"
+        },
+        "mini": {
+          "cost": {
+            "input": "2.5e-07",
+            "output": "2e-06",
+            "cache_read": "2.5e-08"
+          },
+          "provider_type": "openai",
+          "wire_api": "responses"
+        },
+        "small": {
+          "cost": {
+            "input": "2.5e-07",
+            "output": "2e-06",
+            "cache_read": "2.5e-08"
           },
           "provider_type": "openai",
           "wire_api": "responses"

docs/troubleshooting/copilot-model-alias-failure-investigation.md

@@ -0,0 +1,83 @@
+# Investigation: Copilot failure with model alias `small`
+
+## Scope
+
+- Workflow run: https://github.com/github/gh-aw/actions/runs/28986878657
+- Failing job: https://github.com/github/gh-aw/actions/runs/28986878657/job/86018033327
+- Failing step: `Execute GitHub Copilot CLI` (step 24)
+- Workflow file: `/home/runner/work/gh-aw/gh-aw/.github/workflows/smoke-copilot-small.md`
+
+## Observed failure
+
+The run fails before the Copilot harness starts.
+
+```
+[ERROR] Error: model 'small' is unsupported or unrecognized by this AWF version. Did you mean 'gpt-4'?
+```
+
+The error is emitted by `awf` and the step exits with code `1`.
+
+## What code emits this error
+
+- In this repository, the Copilot execution step is generated by:
+  - `pkg/workflow/copilot_engine_execution.go` (sets `COPILOT_MODEL`)
+  - `pkg/workflow/awf_helpers.go` (`BuildAWFCommand`, emits `awf --config ...`)
+- The failing job log shows that exact generated command runs:
+  - `awf --config "${RUNNER_TEMP}/gh-aw/awf-config.json" ...`
+  - with `COPILOT_MODEL: small`
+- The literal message text
+  - `Error: model 'small' is unsupported or unrecognized by this AWF version. Did you mean 'gpt-4'?`
+  is **not** present in `github/gh-aw` source (only in this investigation doc).
+- Conclusion: this string is generated by the external `awf` binary/runtime (from `gh-aw-firewall`), and `gh-aw` is surfacing it via the generated AWF invocation.
+
+## Root cause in `github/gh-aw-firewall`
+
+The failure is caused by validation ordering and data flow inside AWF:
+
+1. `src/commands/validators/config-assembly.ts` reads `COPILOT_MODEL` and calls:
+   - `validateCopilotModel(copilotModel)`
+   - on failure, it logs `validation.message` and exits (`process.exit(1)`).
+2. `src/copilot-model.ts` generates the line-15 failure text (`Error: model 'small' is unsupported or unrecognized by this AWF version...`) and validates against a hardcoded `SUPPORTED_COPILOT_MODELS` set plus a limited retired-alias map.
+3. That validator does **not** read runtime alias config (`config.modelAliases` / `AWF_MODEL_ALIASES`), so aliases such as `small` are not considered valid at this preflight step.
+4. AWF does carry alias config forward (`modelAliases` is assembled and exported as `AWF_MODEL_ALIASES`), but that is for api-proxy model resolution later in the request path, after the failing preflight check.
+
+So `small` fails before alias resolution can run, while concrete models in the allowlist (for example `gpt-5-mini`) can pass this validator.
+
+## Evidence gathered
+
+1. The workflow explicitly sets:
+
+```yaml
+engine:
+  id: copilot
+  model: small
+```
+
+2. The runtime environment confirms:
+
+```text
+COPILOT_MODEL: small
+```
+
+3. The generated AWF config in the failing step includes builtin model aliases, including:
+
+```json
+"small": ["mini"]
+```
+
+## Current hypothesis
+
+`small` is accepted by gh-aw as a valid alias and is emitted into AWF config, but AWF v0.27.27 rejects `COPILOT_MODEL=small` in preflight model validation (`validateCopilotModel`) before alias resolution (`AWF_MODEL_ALIASES`) is applied in the api-proxy path.
+
+## Why this matters
+
+The smoke workflow intended to validate Copilot behavior with a compact alias currently validates an integration mismatch instead, causing a deterministic failure.
+
+## Suggested follow-up validation
+
+1. Re-run the smoke workflow with `engine.model: mini`.
+2. Re-run with `engine.model: gpt-5-mini`.
+3. Compare whether failure only affects alias names vs. concrete provider model identifiers.
+4. If alias handling is not supported in this AWF path, either:
+   - update AWF to a version that resolves aliases in this path, or
+   - keep smoke workflow on a concrete model and add a dedicated alias-compatibility test once AWF support is confirmed.

pkg/cli/data/models.json

@@ -381,7 +381,25 @@
           "cost": {
             "input": "2.5e-07",
             "output": "2e-06",
-            "cache_read": "2.5000000000000002e-08"
+            "cache_read": "2.5e-08"
+          },
+          "provider_type": "openai",
+          "wire_api": "responses"
+        },
+        "mini": {
+          "cost": {
+            "input": "2.5e-07",
+            "output": "2e-06",
+            "cache_read": "2.5e-08"
+          },
+          "provider_type": "openai",
+          "wire_api": "responses"
+        },
+        "small": {
+          "cost": {
+            "input": "2.5e-07",
+            "output": "2e-06",
+            "cache_read": "2.5e-08"
           },
           "provider_type": "openai",
           "wire_api": "responses"