feat(ai-panel): per-workload temperature + endpoint URL UX for local runtimes#2165
Merged
Conversation
…or local runtimes
- Extend provider-string grammar to `<slug>:<model>[@<temp>]`; the Rust
factory parses and strips the suffix, attaching it as a temperature
override on the built OpenAI-compatible provider so the model id sent
upstream stays clean.
- Add a temperature slider (0..2, optional) to the Custom routing dialog so
each workload can pin its own temperature.
- Replace the API-key hack in the Ollama/LM Studio connect dialog with a
proper Endpoint URL field, with localhost defaults and http(s) validation.
Ollama also persists `local_ai.base_url` (and flips the runtime/opt-in
flags) so the Rust factory routes chat to the chosen host.
- Fix the "Connect {label} OpenAI" dialog title by dropping the unsupported
interpolation token from the en string.
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds per-workload temperature override support encoded as ChangesTemperature Override and Provider Connection
Sequence Diagram(s)sequenceDiagram
participant User
participant AIPanel_CustomRouting
participant aiSettingsApi
participant ProviderFactory
participant OpenAiCompatibleProvider
User->>AIPanel_CustomRouting: Enable temp override / save (`@0.2`)
AIPanel_CustomRouting->>aiSettingsApi: serializeProviderRef("ollama:model@0.2")
aiSettingsApi->>ProviderFactory: create provider with temperature_override=0.2
ProviderFactory->>OpenAiCompatibleProvider: with_temperature_override(0.2)
OpenAiCompatibleProvider->>OpenAiCompatibleProvider: use override for requests
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
coderabbitai
Bot
added
the
feature
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
app/src/components/settings/panels/AIPanel.tsx (1)
2078-2090:⚠️ Potential issue | 🟠 Major | ⚡ Quick winClear
kind: 'local'routes when disconnecting Ollama.This branch only resets cloud refs keyed by the local slug. Persisted Ollama routes round-trip back as
{ kind: 'local' }, so after a reload the UI can say “Disconnected” while workloads still point at Ollama. Resetref.kind === 'local'here as well whenlocalKind === 'ollama'.Suggested fix
const nextRouting = Object.fromEntries( Object.entries(draft.routing).map(([wid, ref]) => [ wid, - ref.kind === 'cloud' && ref.providerSlug === localKind + (ref.kind === 'cloud' && ref.providerSlug === localKind) || + (localKind === 'ollama' && ref.kind === 'local') ? ({ kind: 'openhuman' } as const) : ref, ]) ) as typeof draft.routing;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/src/components/settings/panels/AIPanel.tsx` around lines 2078 - 2090, When removing a cloud provider (inside the branch that checks enabled && existing), the routing mapping created in nextRouting should also clear refs that became { kind: 'local' } for Ollama; update the mapping in the Object.entries(draft.routing).map callback used to build nextRouting so the replacement condition is true when either (ref.kind === 'cloud' && ref.providerSlug === localKind) OR (localKind === 'ollama' && ref.kind === 'local'), and then set those entries to ({ kind: 'openhuman' } as const); keep the rest of the logic and the setDraft call as-is.app/src/services/api/aiSettingsApi.ts (1)
134-167:⚠️ Potential issue | 🟠 Major | ⚡ Quick winReserve
ollamafrom the cloud route grammar.
serializeProviderRef({ kind: 'cloud', providerSlug: 'ollama', model })andserializeProviderRef({ kind: 'local', model })both emitollama:..., butparseProviderStringalways reads that back askind: 'local'. That makes the contract non-round-trippable and changes behavior after save/reload. Either rejectproviderSlug === 'ollama'on the cloud path or give local routes a distinct wire prefix.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/src/services/api/aiSettingsApi.ts` around lines 134 - 167, The wire token "ollama:" must be reserved for local providers to preserve round-trip parity: update parseProviderString to keep treating any "ollama:..." input as local (no change) and add validation in serializeProviderRef (and any code that constructs cloud ProviderRef) to reject or disallow a cloud provider with providerSlug === 'ollama' (e.g., throw an error or convert callers to use kind: 'local'), so serializeProviderRef never emits `ollama:` for a cloud ref; modify functions parseProviderString and serializeProviderRef and ensure ProviderRef construction is validated so cloud refs with providerSlug 'ollama' are not allowed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/src/components/settings/panels/AIPanel.tsx`:
- Around line 2296-2315: The current catch for openhumanUpdateLocalAiSettings
silently logs and continues, leaving the UI marked connected while Ollama will
still use the old local_ai.base_url; change this to abort the connect flow
instead: in the block where isLocalRuntime && slug === 'ollama' and you compute
baseUrl, stop swallowing the error in the catch for
openhumanUpdateLocalAiSettings (refer to openhumanUpdateLocalAiSettings,
baseUrl, and the Ollama branch check) — surface the failure by rethrowing the
error (or return a rejected Promise) and ensure the outer handler that adds the
provider/dialog close respects that and does not add the provider or close the
dialog when the update fails so the UI remains unconnected.
---
Outside diff comments:
In `@app/src/components/settings/panels/AIPanel.tsx`:
- Around line 2078-2090: When removing a cloud provider (inside the branch that
checks enabled && existing), the routing mapping created in nextRouting should
also clear refs that became { kind: 'local' } for Ollama; update the mapping in
the Object.entries(draft.routing).map callback used to build nextRouting so the
replacement condition is true when either (ref.kind === 'cloud' &&
ref.providerSlug === localKind) OR (localKind === 'ollama' && ref.kind ===
'local'), and then set those entries to ({ kind: 'openhuman' } as const); keep
the rest of the logic and the setDraft call as-is.
In `@app/src/services/api/aiSettingsApi.ts`:
- Around line 134-167: The wire token "ollama:" must be reserved for local
providers to preserve round-trip parity: update parseProviderString to keep
treating any "ollama:..." input as local (no change) and add validation in
serializeProviderRef (and any code that constructs cloud ProviderRef) to reject
or disallow a cloud provider with providerSlug === 'ollama' (e.g., throw an
error or convert callers to use kind: 'local'), so serializeProviderRef never
emits `ollama:` for a cloud ref; modify functions parseProviderString and
serializeProviderRef and ensure ProviderRef construction is validated so cloud
refs with providerSlug 'ollama' are not allowed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 1d7be75c-9c51-44ed-8f9c-2fc199beb416
📒 Files selected for processing (6)
app/src/components/settings/panels/AIPanel.tsxapp/src/lib/i18n/en.tsapp/src/services/api/aiSettingsApi.tssrc/openhuman/inference/provider/compatible.rssrc/openhuman/inference/provider/factory.rssrc/openhuman/inference/provider/factory_test.rs
`chat_provider` was added to the config schema and UI in tinyhumansai#2152 but the chat-factory's `provider_for_role` was never extended with a matching arm, so the `chat` role fell through to `None` and every chat message silently routed to the OpenHuman backend regardless of the user's configured provider (e.g. `openai:gpt-4`). Verified via `inference-probe --mode raw --role chat`: before: create_chat_provider role=chat resolved_string=openhuman after: create_chat_provider role=chat resolved_string=openai:gpt-4 outbound chat/completions -> https://api.openai.com/v1/chat/completions Add `chat_workload_override_respected` and extend `all_workloads_default_to_openhuman` so the arm can't drop out again.
Swallowing the openhumanUpdateLocalAiSettings error left the UI marked "connected" while chat would keep using the default localhost host — because the Rust factory's Ollama branch reads `local_ai.base_url`, not `cloud_providers[].endpoint`. Let the error propagate to ProviderKeyDialog.handleSave so the user sees the failure and the dialog stays open instead. Caught in CodeRabbit review on tinyhumansai#2165.
…works When the user toggled Ollama on, the chip handler added an `ollama` entry to React's draft `cloud_providers`, but the eager-flush effect filtered it out and the Rust `is_slug_reserved` check would have rejected it anyway. Result: opening the Custom routing dialog and selecting Ollama hit `list_configured_models` which returned "no cloud provider with id or slug 'ollama' found" `ollama` doesn't need to stay reserved — the factory's chat routing short-circuits via the `ollama:<model>` prefix branch before any `cloud_providers` lookup happens, so a synthetic `ollama` entry is benign for chat dispatch and unblocks the model-list dropdown. Drop `ollama` from both `is_slug_reserved` (Rust) and the frontend eager-flush filter, and add regression tests for the reserved-slug set.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
app/src/components/settings/panels/AIPanel.tsx (1)
466-595: 🏗️ Heavy liftExtract the new dialogs out of
AIPanel.This PR adds more self-contained modal state, validation, and submit flow logic to a file that is already ~2.5k lines long. Pulling
ProviderKeyDialogandCustomRoutingDialoginto dedicated modules would make this panel much easier to navigate and test.As per coding guidelines:
**/*.{js,ts,tsx,jsx}: Prefer files ≤ ~500 lines per source file; split modules when growing to maintain readability and single responsibility.Also applies to: 1551-1869, 2260-2325
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/src/components/settings/panels/AIPanel.tsx` around lines 466 - 595, The file AIPanel.tsx contains large inline modal components; extract ProviderKeyDialog (and similarly CustomRoutingDialog) into their own files to reduce file size and improve testability: create ProviderKeyDialog.tsx exporting the ProviderKeyDialog component (keeping props shape, useT hook, state/handlers like handleSave, placeholder logic, and classNames intact), update AIPanel.tsx to import ProviderKeyDialog and remove the in-file definition, and do the same for CustomRoutingDialog (preserve prop types and any referenced helpers like defaultEndpointFor); ensure exports/imports paths are updated and run type checks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/src/components/settings/panels/AIPanel.tsx`:
- Around line 2277-2283: When persisting a local-runtime endpoint (where
isLocalRuntime is true) normalize and validate the URL instead of storing the
raw trimmed value: parse trimmed with new URL(...), throw/reject on parse
errors, and ensure the pathname is not empty or just "/" by appending "/v1" when
needed; then set upserted.endpoint to the normalized URL.toString(). Update the
endpoint assignment in the upsert logic that constructs the CloudProvider (the
block that creates the upserted object using id `p_${slug}_...`, slug, label,
endpoint, authStyle) and apply the same normalization in the other similar
upsert locations noted in the comment (the other blocks that construct
upserted/cloudProviders entries).
- Around line 1819-1841: The temperature slider and number input in AIPanel are
missing accessible names; update the inputs that use the temperature state and
setTemperature handler to include explicit accessible names by either adding
aria-label attributes (e.g., aria-label="Temperature" and
aria-label="Temperature value") or by linking each input to a
visible/visually-hidden <label> via aria-labelledby or id/for pairing; ensure
the labels describe the control and match the inputs that reference temperature
and setTemperature so screen readers announce them correctly.
---
Nitpick comments:
In `@app/src/components/settings/panels/AIPanel.tsx`:
- Around line 466-595: The file AIPanel.tsx contains large inline modal
components; extract ProviderKeyDialog (and similarly CustomRoutingDialog) into
their own files to reduce file size and improve testability: create
ProviderKeyDialog.tsx exporting the ProviderKeyDialog component (keeping props
shape, useT hook, state/handlers like handleSave, placeholder logic, and
classNames intact), update AIPanel.tsx to import ProviderKeyDialog and remove
the in-file definition, and do the same for CustomRoutingDialog (preserve prop
types and any referenced helpers like defaultEndpointFor); ensure
exports/imports paths are updated and run type checks.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d3491fc2-3e36-4b37-aae4-682055a968a2
📒 Files selected for processing (1)
app/src/components/settings/panels/AIPanel.tsx
Two CodeRabbit catches on tinyhumansai#2165: 1. Temperature override slider/number inputs had no accessible name — screen readers were announcing an unnamed slider/spinbutton inside the dialog. Add aria-label to both. 2. Local-runtime endpoints like `http://host:11434` passed validation but were stored verbatim, so the /models probe silently failed until the user manually appended `/v1`. Parse with `new URL(...)`, reject non-http(s), append `/v1` when the path is empty or `/`, and feed the normalised endpoint to both the cloud_providers entry and the local_ai.base_url derivation.
…ure override Coverage gate failed on the previous diff at 41% — well below the 80% diff-cover threshold for tinyhumansai#2165. Bring up the changed-line coverage by exercising the actual new surfaces: - aiSettingsApi.test.ts: @temp parse/serialize round-trips, malformed suffix degrades to model id, two-decimal rounding pinned, non-finite temperatures treated as unset, and the omitted-key contract for pre-existing toEqual snapshots. - AIPanel.test.tsx: Ollama chip → endpoint dialog renders a URL field with a localhost default (not an API-key field); non-http endpoints are rejected and the dialog stays open; a successful save normalises the URL and calls openhumanUpdateLocalAiSettings with the stripped base_url; the Custom routing dialog wires a per-workload temperature override into the saved provider string.
AppearancePanel was entirely un-i18n'd; MascotPanel had a handful of
hardcoded strings (header title, color-swatch aria + labels, character
metadata "states"/"visemes", "Preview ·" caption, library-unavailable
error). Wire both panels to useT and add 21 new keys to en.ts +
chunks/{locale}-5.ts for all 11 locales (non-English values default to
the English text for the translator queue).
Save flow for both the chip toggle dialog (ProviderKeyDialog) and the full editor (CloudProviderEditor) now: 1. Persists the credential (or local_ai.base_url for Ollama). 2. Flushes the new cloud_providers list to disk so the Rust controller can resolve the slug. 3. Calls `openhuman.inference_list_models` (i.e. hits the provider's `/models` endpoint with the configured auth header). 4. On failure: rolls back — restores the prior cloud_providers list and clears the API key — and surfaces the error inline in the dialog so the user can fix the value and retry. The provider never lands in the saved state. OpenHuman backend is exempt (session JWT, no probe-able /models).
The chip toggle + full editor already probe `/models` at add-time, but
the global SaveBar was committing the draft without re-verifying. Now
the save() callback diffs draft vs saved cloud_providers and re-probes
any entry that is new or whose endpoint changed before calling
`saveAISettings`. A probe failure blocks the save and surfaces the
error in the existing top-of-panel alert ("Could not reach X: ...
Settings were not saved.") so a stale/unreachable provider can never
slip into the saved config and start routing chat traffic to a dead
host. OpenHuman backend stays exempt.
…rror field
LM Studio returns HTTP 200 at unknown paths like `/v11/models` with a
JSON body `{"error":"Unexpected endpoint or method..."}` instead of a
4xx. The previous `list_configured_models` happily parsed that as
`data: []` and reported success, so the AI-panel /models probe silently
accepted a typo'd endpoint.
Tighten the response check: any top-level `error` field, or a body
missing the `data` array entirely, is now a probe failure. A valid
empty list (`{"data": []}`) is still accepted — that's the legitimate
"endpoint reached, no models loaded yet" case for fresh Ollama.
Repro on a real LM Studio install:
curl http://localhost:1234/v11/models
→ HTTP 200, {"error":"Unexpected endpoint or method..."}
The lead/orchestrator's `provider_role` was inferred from `config.default_model` — bootstrap pinned that to `reasoning-v1`, so setting `chat_provider` in Settings → LLM → Routing had no effect on the main chat turn (it kept routing to `reasoning_provider`, which most users leave on the OpenHuman backend). Three coupled changes so `chat_provider` actually drives the user-facing chat: - Introduce `MODEL_CHAT_V1 = "chat-v1"` and re-export it; flip `DEFAULT_MODEL` from `reasoning-v1` to `chat-v1`. - session/builder.rs: only the explicit `hint:<role>` form routes to a specialised workload now; everything else (including legacy `reasoning-v1` on disk for existing users) falls through to `chat`. Subagents still set their own role via `ModelSpec::Hint(...)`. - factory::make_openhuman_backend: `hint:chat` now translates to `MODEL_CHAT_V1` (was `MODEL_REASONING_QUICK_V1`), keeping the backend tier name in sync with the new constant. Verified live with chat_provider="lmstudio:google/gemma-4-e4b": [chat-factory] role=chat slug=lmstudio model=google/gemma-4-e4b [provider:cloud] outbound chat/completions -> http://localhost:1234/v1/chat/completions
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
local_ai.base_urlso the Rust factory actually routes chat to the chosen host.<slug>:<model>[@<temp>](both UI parse/serialize and Rust factory).{label}interpolation token fromsettings.ai.connectProvider(was rendering as "Connect {label} OpenAI").Problem
The LLM settings page had three gaps that came up while wiring custom models:
local_ai.base_urlso the Rust factory kept routing tohttp://localhost:11434regardless of what the user typed.default_temperature. Different workloads (reasoning vs. memory summarisation vs. coding) want different temperatures.t()doesn't interpolate.Solution
Frontend (
app/src/services/api/aiSettingsApi.ts,app/src/components/settings/panels/AIPanel.tsx)ProviderRefgains optionaltemperature?: number | nullfor cloud/local kinds.splitModelAndTemp/joinModelAndTemphelpers handle<model>[@<temp>]parse/serialize round-trips.parseProviderStringonly emitstemperaturewhen set, preserving existing test snapshots.ProviderKeyDialognow takes anisLocalRuntimeprop. For local runtimes it renders a proper URL input withdefaultEndpointFor(slug)defaults (http://localhost:11434/v1,http://localhost:1234/v1) and rejects non-http(s)values.local_ai.base_urlviaopenhumanUpdateLocalAiSettings(stripping a trailing/v1sincemake_ollama_providerappends it) and flipsruntime_enabled/opt_in_confirmedin tandem so chat actually uses the chosen host.CustomRoutingDialogadds a checkbox-gated slider + numeric input for temperature.diffSummarysurfaces temperature changes in the unsaved-changes bar.Rust (
src/openhuman/inference/provider/)factory.rsparses@<temp>off bothollama:and<slug>:strings viasplit_model_and_temperature. The clean model id is what reaches the upstream API; the temperature is attached to the constructed provider via a builder method. Malformed@<garbage>is treated as part of the model id rather than silently dropped.compatible.rsaddstemperature_override: Option<f64>andwith_temperature_override(..).effective_temperature()honours the override after the existingtemperature_unsupported_modelsglob filter.i18n
settings.ai.connectProvider:Connect {label}→Connect(concatenation now produces a sane title).Submission Checklist
pnpm test:coverage/pnpm test:rustnot run locally; CIcoveragegate will enforce. Targeted Vitest suites (aiSettingsApi,AIPanel) and Rust factory tests all pass.Impact
<slug>:<model>strings parse identically;@<temp>is opt-in per workload./modelsdiscovery already exists and is unchanged.base_urlis user-supplied, validatedhttp(s)://; secrets remain inauth-profiles.json(no change).Related
CustomRoutingDialoglocal branch still uses the installed-Ollama-models list; switching it tolistProviderModels('ollama')would let a remote Ollama host populate the dropdown.AI Authored PR Metadata (required for Codex/Linear PRs)
Linear Issue
Commit & Branch
fix/local-models31fab6b85284339f51ff98bcff879ae1969a9e48Validation Run
pnpm --filter openhuman-app format:check(ran Prettier write on changed files)pnpm typecheckpnpm debug unit aiSettingsApi(37/37),pnpm debug unit AIPanel(11/11),cargo test … temperature(19/19 incl. 2 new factory tests)cargo check --manifest-path Cargo.toml --bin openhuman-core(only pre-existing warnings)Validation Blocked
command:N/Aerror:N/Aimpact:N/ABehavior Changes
local_ai.base_url.Parity Contract
@<temp>parse and route identically to before.temperature_unsupported_modelsglob filter still applied after the override.Duplicate / Superseded PR Handling
Summary by CodeRabbit