Skip to content

[WIP] Add regression coverage for model propagation in workflow - #50321

Closed
pelikhan with Copilot wants to merge 1 commit into
mainfrom
copilot/squad-plan-add-regression-coverage
Closed

[WIP] Add regression coverage for model propagation in workflow#50321
pelikhan with Copilot wants to merge 1 commit into
mainfrom
copilot/squad-plan-add-regression-coverage

Conversation

Copilot AI commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a plan and make progress.


This section details on the original issue you should resolve

<issue_title>[squad-plan] Add regression coverage for model propagation from workflow frontmatter to aw_info and engine runtime</issue_title>
<issue_description>## Objective
Add focused tests that lock in correct propagation of top-level workflow model through compilation artifacts and runtime configuration.

Issue context

Issue #50312 exposed a likely gap in test coverage: Avenger specified model: claude-haiku-4.5, but audited runs used claude-opus-4-8. Even once the underlying bug is fixed, we need durable regression coverage around model propagation to prevent recurrence.

Implementation guidance

  • Add targeted tests near existing engine/aw_info coverage in pkg/workflow.
  • Cover at least:
    • top-level model on a Claude engine workflow
    • activation job outputs exposing the same model
    • generated aw_info step receiving the expected model value
    • runtime engine config/args not silently swapping to a different default
  • If useful, add a narrow audit-side test only if reporting logic contributed to the mismatch.

Acceptance criteria

  • New tests clearly cover top-level model propagation for the Claude engine path.
  • Tests would catch a future regression where frontmatter says one model but activation/runtime uses another.
  • Test additions are targeted and pass with existing workflow/unit test runners (make test-unit).

Dependency / ordering notes

Best done immediately after or alongside the model-mismatch fix issue. If that investigation reveals the problem is only in reporting (not runtime), tailor these tests to the true failure point.

Generated by 🧑‍🤝‍🧑 Squad Plan · auto · 37.1 AIC · ⌖ 2.24 AIC · ⊞ 9.5K ·
Comment /squad-plan to run again

Comments on the Issue (you are @copilot in this section)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[squad-plan] Add regression coverage for model propagation from workflow frontmatter to aw_info and engine runtime

2 participants