feat(github-bot): UI-driven model selector with inline directive override

[donnfelker]

Summary

• Adds a per-installation model selector to the GitHub bot integration settings UI, with an opt-in toggle that allows @<bot> model: <alias> reasoning: <effort> inline directives in PR comments to override the configured model for a single run.
• Introduces shared model resolution helpers (MODEL_OPTIONS, MODEL_ALIASES, resolveModelAlias, normalizeModelId) in @open-inspect/shared so the GitHub and Linear bots share one source of truth for model ids, aliases, and reasoning-effort compatibility.
• Persists the integration's model and reasoningEffort selection in the control-plane's integration-settings store, plus a boolean allowInlineDirectiveOverride flag.
• Mirrors the inline-directive parser as packages/github-bot/src/inline-directive.ts and a near-clone of the Linear bot's resolver as packages/github-bot/src/model-resolution.ts (kept separate so the two bots' policies can drift).
• Documents the end-to-end "how do I add or remove a model" workflow in docs/MODELS.md.

Screenshots

Summary by CodeRabbit

• New Features

  • Per-comment inline directives (model: / reasoning:) that can override model and reasoning when allowed; directive tokens are stripped from prompts
  • Global defaults for model and reasoning, plus a toggle to enable/disable per-comment directive overrides
  • Friendly model aliases (haiku, sonnet, opus) resolve to canonical models

• Documentation

  • Added comprehensive model configuration and workflow guide

• Tests

  • Expanded test coverage for directive parsing, resolution precedence, and config behavior

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e942cbce-af33-4893-b350-ba6f9b4f19cd

📥 Commits

Reviewing files that changed from the base of the PR and between f010918 and d91284a.

📒 Files selected for processing (1)

• packages/web/src/components/settings/integrations/github-integration-settings.tsx

---

📝 Walkthrough

Walkthrough

Adds inline directive parsing to GitHub comment handling, shared model alias resolution, session-level model/reasoning resolution with directive→config→env precedence, control-plane schema/validation, web UI controls for defaults and directive override toggle, Linear bot alias delegation, tests, and model onboarding docs.

Changes

Inline Directive Override Feature

Layer / File(s)	Summary
Shared Model Alias Resolution packages/shared/src/models.ts	Adds internal MODEL_ALIASES and exported resolveModelAlias() to canonicalize family shortcuts and bare Anthropic versions.
Configuration Schema & Defaults packages/shared/src/types/integrations.ts, packages/github-bot/src/utils/integration-config.ts	Adds allowInlineDirectiveOverride to GitHubBotSettings/ResolvedGitHubConfig, wires fail-closed default to true, and extends fetched config typing/defaulting.
Control-Plane Validation & Resolved Response packages/control-plane/src/db/integration-settings.ts, packages/control-plane/src/routes/integration-settings.ts, packages/control-plane/test/integration/integration-settings.test.ts	Validates allowInlineDirectiveOverride is boolean during normalization and includes the field in resolved config responses with tests for defaulting/propagation.
Inline Directive Parsing packages/github-bot/src/inline-directive.ts, packages/github-bot/test/inline-directive.test.ts	Implements parseInlineDirective() to extract model:/reasoning: tokens (boundary-safe regex), normalize/validate model via shared helpers, validate reasoning compatibility, strip tokens, and return ParsedDirective.
Session Model & Reasoning Resolution packages/github-bot/src/model-resolution.ts, packages/github-bot/test/model-resolution.test.ts	Adds ResolveInput/ResolvedModelSettings and resolveSessionModelSettings() implementing directive → config → env model precedence and reasoning-effort compatibility/nulling rules.
Handler Integration: Directive Parsing & Model Resolution packages/github-bot/src/handlers.ts, packages/github-bot/test/handlers.test.ts	Parses directives in comment handlers, resolves session model settings, uses cleaned directive body as prompt input, creates sessions with resolved settings, and logs directive_applied observability fields; tests cover override vs. stripping behaviors.
Web UI Settings for Defaults & Override Toggle packages/web/src/components/settings/integrations/github-integration-settings.tsx	Adds UI/state for default model, model-driven reasoning effort, and allowInlineDirectiveOverride toggle; updates init, reset, and save payloads.
Linear Bot Model Resolution Refactor packages/linear-bot/src/model-resolution.ts, packages/linear-bot/src/__tests__/pure-functions.test.ts	Delegates label-based model extraction to shared resolveModelAlias() + isValidModel(), removing hardcoded map and updating tests to expect latest alias targets.
Tests & Documentation packages/github-bot/test/*, packages/control-plane/test/*, docs/MODELS.md	Adds comprehensive tests for parsing, resolution, handler integration, config fetching/defaulting, control-plane validation, and documents model onboarding workflows and alias usage.

Sequence Diagram (high-level request flow):

sequenceDiagram
  participant GitHub
  participant Handler
  participant Parser
  participant Resolver
  participant Session
  participant Store

  GitHub->>Handler: event
  Handler->>Parser: comment body
  Parser-->>Handler: ParsedDirective
  Handler->>Resolver: env+config+directive+allowFlag
  Resolver-->>Handler: ResolvedModelSettings
  Handler->>Session: create session
  Session->>Store: persist and log

Loading

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

• ColeMurray
• open-inspect

🐰 A rabbit reads the directives neat,
Resolves the model and trims the feet—
Comments whisper model: opus clear,
The bot obeys when toggles steer,
Hopping logs and tests, the flow complete!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 27.78% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main changes: introducing a UI-driven model selector for the GitHub bot and an inline directive override feature.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/github-bot/src/inline-directive.test.ts`:
- Line 1: Move the test file inline-directive.test.ts out of the package source
folder into the package's test directory (i.e., relocate it from the src tree
into test/), update any relative import paths inside the file to reflect the new
location, and ensure the package's test discovery (vitest/glob or package.json
test config) still includes test/*.test.ts so this file is picked up; no code
changes to tests themselves are required beyond fixing imports/paths.

In `@packages/github-bot/src/model-resolution.test.ts`:
- Line 1: The test file model-resolution.test.ts is placed under src/ but should
live in the package test folder; move the file from
packages/github-bot/src/model-resolution.test.ts to
packages/github-bot/test/model-resolution.test.ts and update any import paths if
necessary (ensure the vitest imports: describe, it, expect remain valid and any
relative imports inside model-resolution.test.ts still resolve after
relocation).

In `@packages/github-bot/src/model-resolution.ts`:
- Around line 64-66: The current early return when modelFromDirective is true
always sets reasoningEffort: null and prevents the config fallback; change the
logic so that if modelFromDirective is true you only return immediately when the
directive explicitly provided a reasoning value (use that value for
reasoningEffort), otherwise do not override reasoningEffort and allow the
subsequent config fallback to run. Specifically, adjust the branch around
modelFromDirective to check for the presence/definedness of the directive's
reasoning (instead of unconditionally assigning null), returning { model,
reasoningEffort: <directiveReasoning> } only when that reasoning is present.

In `@packages/shared/src/models.ts`:
- Around line 206-223: The fallback in resolveModelAlias passes the original raw
string to normalizeModelId which does case-sensitive prefix checks, so
mixed-case inputs like "GPT-5.4" bypass normalization; update resolveModelAlias
to call normalizeModelId with the already-lowercased variable (use lower instead
of raw) so normalizeModelId sees canonical lowercase prefixes and correctly
normalizes bare "claude-"/"gpt-" ids.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1e99d115-f7b6-4a19-a131-76e0f6118ff1

📥 Commits

Reviewing files that changed from the base of the PR and between 4f8993a and e27b56b.

📒 Files selected for processing (17)

• docs/MODELS.md
• packages/control-plane/src/db/integration-settings.ts
• packages/control-plane/src/routes/integration-settings.ts
• packages/control-plane/test/integration/integration-settings.test.ts
• packages/github-bot/src/handlers.ts
• packages/github-bot/src/inline-directive.test.ts
• packages/github-bot/src/inline-directive.ts
• packages/github-bot/src/model-resolution.test.ts
• packages/github-bot/src/model-resolution.ts
• packages/github-bot/src/utils/integration-config.ts
• packages/github-bot/test/handlers.test.ts
• packages/github-bot/test/integration-config.test.ts
• packages/linear-bot/src/__tests__/pure-functions.test.ts
• packages/linear-bot/src/model-resolution.ts
• packages/shared/src/models.ts
• packages/shared/src/types/integrations.ts
• packages/web/src/components/settings/integrations/github-integration-settings.tsx

[coderabbitai]

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Model-only directives currently null out reasoning unintentionally

On Line 64, modelFromDirective triggers reasoningEffort: null even when no directive reasoning was supplied. That skips the Line 69 config fallback and is broader than the comment on Lines 62-63.

Proposed fix

-  if (modelFromDirective) {
+  if (modelFromDirective && input.directiveReasoningEffort) {
     return { model, reasoningEffort: null };
   }

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	if (modelFromDirective) {
	return { model, reasoningEffort: null };
	}
	if (modelFromDirective && input.directiveReasoningEffort) {
	return { model, reasoningEffort: null };
	}

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/github-bot/src/model-resolution.ts` around lines 64 - 66, The
current early return when modelFromDirective is true always sets
reasoningEffort: null and prevents the config fallback; change the logic so that
if modelFromDirective is true you only return immediately when the directive
explicitly provided a reasoning value (use that value for reasoningEffort),
otherwise do not override reasoningEffort and allow the subsequent config
fallback to run. Specifically, adjust the branch around modelFromDirective to
check for the presence/definedness of the directive's reasoning (instead of
unconditionally assigning null), returning { model, reasoningEffort:
<directiveReasoning> } only when that reasoning is present.

[donnfelker]

Declining this change — the current behavior is intentional and is covered by an explicit test in model-resolution.test.ts ("directive model with no directive reasoning: drops config reasoning entirely"). The design choice: when a user supplies a fresh model via inline directive, we deliberately do not carry the previous config reasoning over, because reasoning is tightly coupled to the model and the override is treated as a clean reset. Adopting the suggested fallback would require updating both the test and the surrounding doc comment to flip the documented contract.

[coderabbitai]

@donnfelker, understood — thank you for the detailed explanation. The "clean reset" contract makes good sense: since reasoning effort is model-specific, inheriting config reasoning when the user explicitly switches models via directive could silently apply an incompatible or unexpected effort level. The existing test case documents this contract clearly, so no change is needed here.

(　´･‿･｀) ( )

---

✏️ Learnings added

Learnt from: donnfelker
Repo: ColeMurray/background-agents PR: 619
File: packages/github-bot/src/model-resolution.ts:64-66
Timestamp: 2026-05-14T19:35:18.813Z
Learning: In `packages/github-bot/src/model-resolution.ts`, when a user supplies a model via an inline directive (`modelFromDirective = true`) but provides no directive reasoning effort, the resolver intentionally returns `reasoningEffort: null` and does NOT fall back to `configReasoningEffort`. This is by design — a directive model override is treated as a "clean reset" because reasoning effort is tightly coupled to the model. This contract is covered by the test "directive model with no directive reasoning: drops config reasoning entirely" in `packages/github-bot/src/model-resolution.test.ts`.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.