LLM Tool Platform

[balloob]

This proposal is part of a multi-step plan to upgrade our LLM infrastructure.

Context

Home Assistant exposes capabilities to LLMs as tools. Today these tools are
collected centrally in homeassistant/helpers/llm.py: AssistAPI._async_get_tools
hardcodes the set (every registered intent, one tool per script, plus the
calendar/todo/datetime/live-context tools). This has three problems:

1. Tool code does not live with the integration it belongs to. A calendar tool
   lives in helpers/llm.py, not in calendar. There is no first-class way for an
   integration to say "expose this capability to the LLM." The only extension point
   is llm.async_register_api, which registers a whole parallel API (used by MCP) and
   is too heavy for contributing a single tool.

2. Every intent is blanket-wrapped as a tool. As intents and integrations grow,
   the tool count grows without bound. Large language models lose tool-selection
   accuracy once they have more than a few dozen tools, and local models with small
   context windows cannot hold every tool definition at all. We need a curated set,
   not the whole catalogue.

3. Large capability collections cannot be composed. A future surface for
   managing Home Assistant (config entries, areas, automations, and each
   integration's own settings) should be assembled from many integrations, each
   owning its part — not maintained as one central monolith.

This ADR covers v1: a platform that lets integrations register tools, owned by a
new llm integration, with the existing Assist API as the only surface. It is the
first step of a larger effort (later phases add tool search for large tool counts and
human-in-the-loop approval); those are explicitly out of scope here, but v1 is
designed so they can be added without breaking the registration contract.

Decision

A new llm integration owns a tool platform

We mirror the existing intent split, which has both a helper
(homeassistant/helpers/intent.py) and an integration
(homeassistant/components/intent/). The integration's async_setup discovers
per-integration platform files and registers the built-ins.

• homeassistant/helpers/llm.py remains the framework (API, APIInstance, Tool,
  MergedAPI, the registries) and gains the per-tool registry below.

• A new homeassistant/components/llm/ integration owns lifecycle. Its async_setup
  registers the built-in AssistAPI (moving the bootstrap out of the helper, where it
  is currently created lazily) and discovers tool platforms:

  async def async_setup(hass: HomeAssistant, config: ConfigType) -> bool:
      llm.async_register_api(hass, llm.AssistAPI(hass))
      await async_process_integration_platforms(hass, DOMAIN, _process)  # DOMAIN = "llm"
      return True
  
  async def _process(hass: HomeAssistant, domain: str, platform) -> None:
      await platform.async_setup_tools(hass)

• An integration contributes tools through a <integration>/llm.py platform file
  exposing async def async_setup_tools(hass) (mirroring intent's
  async_setup_intents). The domain is llm, so there is no clash with the
  <integration>/conversation.py conversation-entity platform.

• conversation and the LLM provider integrations depend on llm, so AssistAPI is
  always available.

Registration is a provider callback into one or more API surfaces

Tools are generated per request today — intents are filtered by exposed domains,
there is one ScriptTool per script, calendar/todo tools appear only if those domains
are exposed. So the registration primitive is a provider callback evaluated per
request with the LLMContext; a static convenience wraps it for fixed tools.

A provider returns its tools and an optional prompt fragment. Today the Assist
system prompt is built conditionally on the tool set (_async_get_api_prompt adds
live-context guidance only when that tool is present, a no-timer note only when timer
intents are absent, and the calendar/todo tools bake exposed entity names into
themselves). That tool-dependent prompt must move with the tools, or the prompt goes
stale once tools live in their integrations.

# homeassistant/helpers/llm.py
@dataclass(slots=True)
class LLMTools:
    tools: list[Tool]
    prompt: str | None = None      # appended to the surface's api_prompt when present

type LLMToolProvider = Callable[[HomeAssistant, LLMContext], LLMTools]

@callback
def async_register_tool_provider(
    hass: HomeAssistant,
    provider: LLMToolProvider,
) -> CALLBACK_TYPE: ...

# homeassistant/components/calendar/llm.py
async def async_setup_tools(hass: HomeAssistant) -> None:
    llm.async_register_tool_provider(
        hass, _calendar_tools
    )

@callback
def _calendar_tools(hass: HomeAssistant, llm_context: LLMContext) -> llm.LLMTools:
    # empty LLMTools when no calendar entity is exposed
    return llm.LLMTools(tools=[CalendarGetEventsTool(...)], prompt=...)

AssistAPI.async_get_api_instance iterates the providers registered for
LLM_API_ASSIST, calls each with the context, collects their tools, and builds
api_prompt from its own core prompt (exposed-entity overview, area,
device-control usage) plus each active provider's prompt fragment — the same work
_async_get_tools + _async_get_api_prompt do today, sourced from the registry
instead of hardcoded, keeping the prompt in lock-step with the tool set.

Built-in tools move into their integrations

The tools currently in helpers/llm.py move to the integration that owns the
capability and register themselves through the platform:
CalendarGetEventsTool → calendar, TodoGetItemsTool → todo, ScriptTool →
script, GetLiveContextTool/GetDateTimeTool → homeassistant/conversation.

Intents are no longer auto-wrapped

AssistAPI._async_get_tools stops blanket-wrapping every registered intent. Exposing
an intent as a tool becomes opt-in:

• The core intents (HassTurnOn, HassLightSet, …) are registered explicitly by
  their owner via a provider that applies today's exposed-domain filter, so current
  Assist behaviour is preserved.
• intent_script becomes a tool platform that registers the user's YAML-defined
  intents. For backwards compatibility these default to exposed in Assist
  (matching today's behaviour, so existing custom intents do not silently disappear),
  with a per-intent opt-out.

Out of scope for v1

No importance/tool-search, no approval policy, no API surfaces beyond Assist (so no
merging or namespacing), and no schema-from-signature introspection (tools stay
llm.Tool classes with voluptuous schemas).

Consequences

• Tool ownership is decentralised. Integrations own their LLM tools in
  <integration>/llm.py, and a future management surface can be composed from many
  integrations contributing into a shared API id — without central edits.
• Migration is required to avoid regressions. Because intents are no longer
  auto-wrapped, every intent currently exposed to the LLM must be re-registered
  explicitly (core intents in v1; integration intents per integration) or it
  disappears from the assistant. This is the main risk and the bulk of the v1 work; it
  is mechanical but broad. A transitional provider that wraps still-unmigrated intents
  can bridge the gap if needed.
• Tool-dependent prompt moves with the tools. The Assist system prompt is today
  assembled conditionally on the active tools. Each provider now carries its own prompt
  fragment, so moving a tool into its integration must move its prompt guidance too;
  AssistAPI keeps only the core prompt and concatenates the active fragments.
• A new always-loaded integration. llm joins the dependency graph (like
  intent). AssistAPI moves from lazy creation in the helper to explicit
  registration at setup.
• The registration contract is forward-stable. The apis mapping and the
  provider-callback shape are chosen so later phases (importance-based selection,
  approval) extend the system without breaking integrations that adopt v1.
• External/MCP APIs are unaffected. async_register_api remains for whole-API
  registration; the new per-tool path is additive.

[balloob]

Implementation update

The first step is up for review:

• Core: Add LLM integration core#174253 — the new llm integration and its tool platform (plumbing only; not consumed by Assist yet)
• Developer docs: Document contributing LLM tools via the llm integration platform developers.home-assistant#3201
• User docs: Add LLM integration documentation home-assistant.io#46325

A few things landed differently than written above, refined while building it:

1. No tool-provider registry / apis mapping yet. Since v1 has only the Assist surface, there is no need for membership-by-apis or an imperative provider registry. Integrations register nothing; the llm integration discovers <integration>/llm.py and pulls from it directly. async_register_tool_provider and the static-tool convenience are gone for now — they can come back when a second surface or importance-based selection actually needs them.

2. Lazy, pull-based platform instead of eager registration. The platform hook is async_get_tools(hass, llm_context) -> LLMTools rather than async_setup_tools(hass). Platforms load through the new LazyIntegrationPlatforms helper, so an integration's llm.py is imported only when its tools are actually requested — not for every integration at startup. Tools and prompt fragments are merged sorted by domain, so the order is independent of integration load order.

3. AssistAPI bootstrap not moved in this PR. #174253 is just the platform plumbing and is not wired into Assist yet. Registering AssistAPI at setup (out of the helper's lazy creation) and having Assist pull its tools from the platform come in the follow-up PRs, together with moving the built-in tools into their integrations.

Everything else — the prompt fragment travelling with the tools, intents becoming opt-in, and the built-ins moving into their integrations — is unchanged.

[emontnemery]

Approved by the core architecture meeting 2026-06-18 with these remarks:

• Always a tool provider, skip direct tool registration
• Do not specify the API to register for