Skip to content

Utility-model safety: no silent flagship fallback, deterministic summaries by default #381

Description

@psamiton

Problem

aiCompleteUtility() (packages/daemon/src/lib/ai-service.ts:149-155) is the funnel for all background LLM calls: per-turn summaries, hourly/daily/weekly/monthly meta-summaries, system-chat fallback. When ai.utilityModel is unset, aiComplete() falls back to autoSelectModel()the first enabled model (:481-490). So on any install where the admin never set a utility model, the highest-frequency background call in the system (a summarization on every substantive mind turn, summarizer.ts:331) silently runs at flagship prices.

All callers already degrade gracefully when aiCompleteUtility returns null: turn summaries and period summaries fall back to deterministic text (summarizer.tsbuildTurnDeterministicSummary, buildPeriodicDeterministicSummary), so there is no correctness dependence on the fallback.

Implementation plan

  1. Remove the silent fallback: aiCompleteUtility returns null unless utilityModel is explicitly configured — do not auto-select. Turn/period summaries become deterministic on unconfigured installs; that's the intended default.
  2. Be loud about it:
    • Setup / models UI: when the admin enables models, auto-suggest the cheapest enabled model (haiku/mini/flash-class) as the utility model, one click to accept.
    • Settings page: persistent hint when unset — "Background summaries are running in basic (non-AI) mode. Pick a utility model for richer summaries."
    • One-time daemon log warning on first utility call with no model.
  3. Verify the system-chat fallback path (packages/daemon/src/lib/chat/system-chat.ts:186, generateSystemReply): confirm the null path produces a reasonable canned reply when a mind DMs @volute with the spirit down and no utility model set.
  4. Trim summarizer inputs (independent savings even with a utility model configured), in buildTranscript (summarizer.ts:291-329):
    • inbound content is currently untruncated — cap at ~500 chars like responses;
    • cap total transcript length (e.g. ~8k chars, keeping head + tail) for tool-heavy turns.
    • (Tool calls are already compact — summarizeTool renders [name arg≤60chars] — no change needed there.)

Non-goals

Do not gate/skip LLM summaries for short turns when a utility model is configured — hierarchical meta-summaries build on turn summaries, and deterministic stubs mixed into the hierarchy degrade the rollups ("Received message. Sent response." tells the daily summary nothing).

Testing

  • Unit: aiCompleteUtility with unset model returns null without touching providers; summarizer produces deterministic summaries in that state; transcript truncation caps respected.
  • Existing summarizer tests keep passing with a configured model.

Part of the token-efficiency series. Principle: never spend flagship tokens by accident.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High priority: real damage or blocks a trainarea:mind-experienceContinuity, transparency, notices, token economicsbugSomething isn't working

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions