Skip to content

[Feat] Allow choosing a model for custom automations - #987

Merged
mrubens merged 5 commits into
developfrom
feat/custom-automation-model
Aug 2, 2026
Merged

[Feat] Allow choosing a model for custom automations#987
mrubens merged 5 commits into
developfrom
feat/custom-automation-model

Conversation

@mrubens

@mrubens mrubens commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

What changed

  • Add an optional per-automation model override to custom automations, editable in the Automations settings editor via the same model picker the ad-hoc task launcher uses, with a leading "Deployment default" option.
  • Persist the override in a new nullable custom_automations.model column (validated provider/model format) and show it on each automation row.
  • Apply the override at launch through the existing harness model override plumbing (task.harness + payload.harnessModelOverrides), shared with the Slack !eval launcher and the programmatic task-launch API.
  • Accept model in the manage_custom_automations MCP tool and REST handler: set on create, preserve when omitted on update, and clear with an explicit null.
  • Degrade gracefully at run time: a persisted override that no longer parses is logged and ignored so the scheduled run still launches on the deployment default instead of failing.

Why this change was made

Custom automations are recurring workloads where the cost/quality tradeoff is known per job: a nightly summary can run on a cheap model while a weekly security audit warrants the strongest one. That variance lives per automation, so a deployment-wide default with a per-automation override fits better than a single global choice.

Impact

Admins can pin a model per custom automation from the web editor or through Roomote MCP, while every existing automation keeps following the deployment default until one is chosen. The override reuses the launch-time harness override path, so run behavior is identical to launching an ad-hoc task with that model. Older application versions ignore the new nullable column, preserving the N-1 rollback guarantee.

@roomote-community

roomote-community Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

No code issues found. See task

  • apps/api/src/handlers/custom-automations/index.ts:42,54 The REST schemas only validate model length. A malformed model such as "not-a-model" reaches createCustomAutomation/updateCustomAutomation, whose format check throws without being translated to a client error, so MCP callers receive a 500 rather than a 400 validation response. Add the same provider/model refinement at this boundary (or map the domain validation error to 400).

Reviewed c301599

@mrubens
mrubens merged commit b5dce68 into develop Aug 2, 2026
18 of 19 checks passed
@mrubens
mrubens deleted the feat/custom-automation-model branch August 2, 2026 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant