Home Assistant
Replies: 9 comments 15 replies
|
As a developer, I think there is no rational for not adding this option with maybe the exception that Open-AI has some sort of exclusivity contract with Nabucasa or that Nabucasa is planing to release this feature under their paid subscription at a later date. I looked over all of the available PRs and the reasoning, as somebody else put it, is just "NO"; Keep in mind that most, if not all, open source software that has integrated LLM's has the ability to set a custom base URL for Open-AI as this is critical for customizing the LLM instance, this is definitely not standard. |
|
I think this is where two different concepts get mixed together. Open source absolutely empowers users with choice, but it does not imply that every possible choice must be supported inside a core integration. Home Assistant already gives users a lot of freedom through custom components, forks and third party integrations. What the core team is trying to protect here is not user choice, but clear ownership and long term maintainability of core integrations. Saying “let the user decide if limitations are acceptable” sounds reasonable in theory, but in practice it does not work that way. When something breaks after an update, users do not file issues against “their choice”, they file issues against Home Assistant. At that point the project implicitly takes responsibility for providers and implementations it does not control. We have concrete precedent for this. With AWS S3, users made the same argument: many services are S3 compatible, so why not allow custom endpoints and let users decide. In reality, when AWS introduced changes, Home Assistant had to follow AWS to keep the official integration working. S3 compatible providers then broke in different ways, and those breakages still landed as Home Assistant bugs. This is exactly why Backblaze B2 ended up as a separate integration, even though it offers S3 compatible endpoints. The same logic applies here. The official A vendor independent or generic AI integration absolutely makes sense, but as a separate integration that explicitly targets OpenAI-compatible APIs and clearly documents that compatibility may vary. That preserves user choice without compromising the stability guarantees of the core OpenAI integration. It’s also worth pointing out that this problem is not hypothetical. There are already several third party integrations that explicitly target OpenAI-like or OpenAI-compatible APIs and make different trade-offs outside of core Home Assistant, for example:
One of these projects even contains a maintainer note that perfectly illustrates the exact issue being discussed here. It started as a simple fork of the built-in OpenAI Conversation integration with only a configurable base URL, but diverged once Home Assistant moved forward with new features such as the Responses API. At that point, keeping compatibility with multiple “OpenAI-compatible” providers required additional development effort, testing, and provider-specific handling. The maintainer explicitly notes that this divergence is the reason why long term maintenance becomes challenging, especially when the integration is no longer used personally by the maintainer. This is a concrete example of why the core team is cautious about absorbing this responsibility into the official OpenAI integration. These integrations clearly demonstrate that user choice already exists within the Home Assistant ecosystem. At the same time, they also show why the core project prefers to keep vendor-independent or compatibility-focused solutions separate from core integrations that are designed around a single, well defined API contract. Your DIY solution demonstrates that this is technically possible. The remaining question is not “can it be done”, but “where does it belong”. From a project perspective, keeping that flexibility outside the core integration is a deliberate and consistent design choice, not a rejection of open source principles. |
|
I think this reply actually highlights the core of the disagreement quite well, so let me respond to it directly. First, regarding responsibility. In theory, yes, compatibility with a protocol falls on the provider that claims to implement it. In practice, that distinction disappears the moment something breaks after a Home Assistant update. Users do not open issues against OpenRouter, Groq or “a provider that should have been compatible”. They open issues against Home Assistant, because that is the integration they updated. From the project’s perspective, adding a base URL option means implicitly accepting that support burden, regardless of where responsibility “should” lie. Second, this is not about a vague “maybe something breaks in the future”. We have already seen this happen multiple times. The AWS S3 case is not theoretical, and the recent move to the OpenAI Responses API is another concrete example. As soon as the official provider evolves, the core integration must follow. At that point, anything that only partially mimics the API either diverges or requires provider specific conditionals. That is not hypothetical risk, it is a repeated pattern. Third, calling this “just a static string parameter” understates the problem. A base URL is not just a transport detail. It defines which API contract you are actually talking to, which features exist, which authentication and headers are expected, and which future changes must be supported. Once that parameter exists in a core integration, every architectural change has to consider an unknown number of backends that the core team does not control. Regarding the point that Home Assistant users are technically inclined and know what they are doing: many are, but the project is explicitly not designed around that assumption. Core integrations are expected to be boring, predictable and supportable at scale. Optional footguns still become support problems once they live in core. The frustration around third party integrations breaking is completely understandable. However, that frustration actually reinforces the core team’s position rather than weakening it. If forks and custom integrations struggle to keep up with rapid changes in the conversation and voice stack, that is exactly why maintainers are reluctant to multiply that maintenance burden inside core. The fact that those forks break frequently is not an argument to absorb them, it is evidence of how costly that compatibility layer really is. Finally, yes, the feature is clearly desired. But demand alone does not determine where a feature belongs. Home Assistant has consistently chosen to place vendor specific or compatibility focused solutions outside of core when they introduce unclear ownership or long term maintenance risks. A generic or OpenAI-compatible AI integration makes sense, but as a separate integration with explicitly defined expectations and trade-offs, not as an extension of the official At this point, the disagreement is less about technical feasibility and more about project boundaries. The question is no longer “can this be added”, but “which part of the ecosystem should own the consequences”. From the project’s track record, keeping this flexibility outside core is a deliberate design choice, not an oversight. Just to be clear, I’m not a maintainer of HA and I’m not speaking on behalf of the core team. However, I do understand the values and constraints they consistently apply across the project, and I think it’s important to articulate that perspective clearly in this discussion. I also noticed the significant interest expressed through reactions in the original thread, which suggests that this topic resonates with many users. That’s precisely why I wanted to take some time to explain how the maintainers tend to reason about these decisions, based on past discussions and concrete precedents in Home Assistant. The goal here is not to shut down the conversation, but to clarify why the project draws the boundaries where it does. |
|
Here is my final take on this. The reasons why maintainers are struggling is because the core implementation changes. openai_conversation does not work in a vacuum inside Home Assistant and if you look at the top issues in all of the above forks you linked, you will see that most of them are not caused by a change in the API provider but due to refactoring in HA. That is because the openai_conversation and the voice workflows are being actively worked on, that makes it a nightmare to fork and maintain. I am using at least 30 other open source project that have implemented OpenAI, and all of those projects have the ability to change the API base path. Even the official client for OpenAI supports that. You can still stick to the official OpenAI specs and also allow for the API base path to change. I would argue that the OpenAI implementation is incomplete based on the fact that it does not align with the functionality of official provided solutions endorsed by OpenAI. In terms of ownership, here is how i stand: I can't agree with the argument that allowing custom URL's for providers implies support for all providers for a particular service across the web. I can give you multiple examples of core integration that support custom URLS like qTorrent, Broadlink, Plex and so many other where the URL of the API endpoint is editable and i could point it to any compatible API, then open a ticket here when it breaks. I get it that in practice people might open a ticket for unsupported providers but here we are with multiple PR's, discussions and tickets opened for this feature... And there will continue to be, as this is required in a myriad of different use-cases. "Dammed if you do, dammed if you don't" I guess :) In the end i think this goes back to the freedom of the user to customize their own instance and not be boxed in to a particular LLM's provider if they choose to run HA. PS: -- My particular use-case is that HA works as an Orchestrator for all of my Home Assistant Voice Preview devices. I think the long-term solution for me might just be to hack all of the Home Assistant Voice Preview devices and point them to a emulated Wyoming-compatible server that mimics HA and scrap HA as an orchestrator altogether since it's simpler then fixing it every month. |
|
I think this reply helps clarify where the real disagreement is, and it’s less about technical feasibility and more about project boundaries and expectations. You’re absolutely right that a lot of breakage in third party forks comes from refactoring inside Home Assistant rather than from upstream API providers. The conversation and voice stacks are evolving quickly, and that makes forks hard to maintain. But that fact actually cuts both ways. It shows why the core team is very careful about expanding the responsibility surface of a core integration. If internal refactors already require coordinated changes, adding an open ended set of external backends makes that coordination problem strictly harder, not easier. On the point that many other projects allow changing the OpenAI base URL: I agree, and that works well in projects where the OpenAI integration is a thin client or a standalone library. Home Assistant’s situation is different. This is also where the distinction between API emulation and protocol support becomes important. Home Assistant is not generally opposed to configurable base URLs when there is a clearly defined protocol behind them. A good example is the Model Context Protocol (MCP), which Home Assistant supports precisely because it has an explicit specification, versioning, feature negotiation and defined extension points. That gives the project a stable contract to build against, even as implementations evolve. “OpenAI-compatible” APIs do not provide that kind of contract. They are implementations attempting to mimic another vendor’s API, which itself evolves unilaterally. There is no independent specification, no formal version negotiation and no guarantee that two providers claiming compatibility behave the same way. Supporting a base URL in that case implicitly commits the core integration to chasing a moving target across multiple providers. Regarding the comparison with integrations like qTorrent, Broadlink or Plex, there is also an important structural difference. Those integrations target a single, well defined server implementation owned by a specific project or vendor. The API surface and behavior are controlled by that one upstream. If, in the future, multiple third party projects started mimicking those APIs with partial or divergent compatibility, it is entirely reasonable to expect that Home Assistant maintainers would revisit their stance and potentially refuse to support those clones as well. That is exactly the situation with “OpenAI-compatible” APIs today. The issue is not the existence of a configurable URL, but the lack of a single authoritative implementation or specification behind it. Regarding ownership, I understand the argument that allowing custom URLs does not automatically imply support for every provider. In practice, however, core integrations set user expectations. Once a feature exists in core, issues will be opened against Home Assistant regardless of disclaimers. This has been demonstrated repeatedly in the project’s history. Ultimately, I think we agree on the goal of avoiding unnecessary vendor lock in and giving users flexibility. Where we differ is on where that flexibility should live. From the Home Assistant project’s perspective, placing that flexibility in separate, explicitly compatibility focused integrations preserves user freedom without turning the core OpenAI integration into a moving target with unclear ownership. |
|
I think it’s also useful to look at how similar decisions have been explained elsewhere in the project. That same principle applies here. This is not about ignoring user needs or bad intentions, but about consciously choosing where the core project draws its boundaries. The discussion around AI integrations fits very naturally into that broader philosophy. It’s also worth noting that this discussion started with assumptions about intent, including speculation about Nabu Casa motivations. Looking at consistent decisions across unrelated parts of the project, it becomes clear that this is a long-standing design approach rather than a reaction to a single feature or vendor. |
|
I’d like to revisit this topic from a slightly different angle, building on the previous discussion. One important point that hasn’t been emphasized enough is that a configurable From Home Assistant’s perspective, this approach has some clear advantages:
This aligns well with how Home Assistant has historically approached similar problems: solving the majority of use cases in a robust and maintainable way, without turning core integrations into compatibility layers for an open-ended set of providers. It also addresses the original use cases raised in this thread (proxy APIs, access to multiple models, experimentation) while keeping clear ownership boundaries inside the project. I’m not suggesting this is the only possible solution, but it seems like a pragmatic alternative that fits the project’s long-term maintenance philosophy, especially given that “OpenAI-compatible” is not a formal protocol with versioning or guaranteed semantics. |
|
OpenAI-compatible is the most used API in the eco system on the self hosted side now. Maybe it's time to evaluate this again? |
This is a bit like an integration that's "Cloud Polling" having a new alternative option that's "Cloud Push," when the users are asking for Local Poll or Local Push. Yes, it's a bit better, and it's a step in the right direction, but it still doesn't allow local-first functionality. The first sentence on the Home Assistant site and the associated google preview (emphasis mine):
The arguments for keeping the OpenAI integration as "OpenAI the company, not the protocol that shares the name" are solid; I honestly think the debate about this would be different if they hadn't become the defacto standard for this. I'm glad Open Responses exists, and it's being driven & getting support from a lot of major companies:
The description of Open Responses from their site kinda encapsulates the situation, re: OpenAI being the driving force & defacto standards creator, though (emphasis again mine): Open Responses is an open-source specification and ecosystem for building multi-provider, interoperable LLM interfaces based on the OpenAI Responses API. It defines a shared schema, and tooling layer that enable a unified experience for calling language models, streaming results, and composing agentic workflows—independent of provider. |
|
OpenResponses is a step in the right direction, but unfortunately it is not yet a mature project e.g. official SDK is missing. We already discussed that project in this thread. |
|
I don't understand how this is substantially different to using Ollama or OpenRouter tooling; beyond maybe improving the discoverability or explaination of what these are. IE: Each allows multiple models to be selected, provides a consistent API (not just to home assistant) and has corresponding integrations with Home assistant? Example: Configuring a model available through Ollama: |
|
Having the simple option to use base URL solves most issues while having some kind of standard. That would allow options such as litellm and similar. Having ai tasks tied to only a tech gigantic seems to be tge exact opposite of what the rest of home assistant is created for. |
|
How is this even a conversation? Setting a base URL for an endpoint is an absolute basic feature in an open source tool. There simply is no rationale that makes since for avoiding it since even the OpenAI API client supports it. |
|
I am confused. I have no idea how the home assistant team is unable to implement this. Other self hosted projects implement that before even thinking about other stuff, but hey, there is an openrouter integration. There is still no simple and easy way to connect my local litellm proxy with my local home assistant? WTF guys? |
|
It's possible, but it's not part of the core.It's possible, but it's not part of the core, so issues are handled by someone who has more ability to ensure compatibility with various service providers. You can look at the following repositories: |
|
how about integrating https://www.openresponses.org ? |
|
This seems like the missing piece. We can implement this specification and be confident that all services are compliant. |
|
While I am now using Extended OpenAI Conversstion so I can change the base URL, for anyone who wishes to change it in core, for instance by using Example here: https://github.com/openai/openai-python?tab=readme-ov-file#configuring-the-http-client |
|
In order to allow more flexibility in using any open compatible API endpoint for ai task and conversation I created this: I have been testing it for a while and so far so good. but my development knowledge of HA is at a basic level. If anybody out there is willing to join the efforts, I want this package to be provider agnostic, using the OpenAI standard, and leaving for the rest the choice to the user what API and models are used! :-) |
I have essentially the same goal with https://github.com/skye-harris/hass_local_openai_llm |
|
@skye-harris : tried your integration, works like a charm! thanks for sharing, it's much more advanced, with the UI than my version, and I see there's already much support for it, so I will not be developing my solution further, as this does exactly what I needed! |
Uh oh!
There was an error while loading. Please reload this page.
Describe the feature
Currently the list of core AI components available for conversations and AI Task support are very limited and tied to dedicated services. This is not in the philosophy of HA, where there should be choice and options to run whichever service you want.
I would propose creating a 'generic' AI component that works both as conversational agent and brings AI Task support to HA.
As this would be the generic variant, you would be able to configure any OpenAI compatible API endpoint.
This could, maybe in time, even replace all separate components using OpenAI API structure.
Use cases
I have 3 profound use cases for this
1 - Usage of proxy API's, there are both public and private services that offer foundation model access through a proxy API.
2 - Usage of agents, with a generic component, you could use openAI compatible 'custom GPT' which are build and maintained outside of HA. Agents like this could have access to an extend system prompt and source library to perform very specific request with AI Task (E.g. energy management, climate control, ...)
3 - local experimentation, to connect custom stacks/agents for test, or even more high level applications such as LM studio
Anything else?
No response
All reactions