Replies: 6 comments 14 replies
|
We've discussed this proposal in the Architecture meeting. We have these questions:
|
|
As part of Month of "What the heck?!" 2024 I added a similar request WTH Forecasting, not only weather but also dynamic energy prices, traffic, etc I just wanted to add to this discussion that there should also be automation triggers, like Electricity Price below X for the next 2 hours, When precipitation is 0 for the next 30 minutes and alike |
|
This is a feature I am also really hoping to see implemented soon. However, I have some thoughts about the implementation laid out in this proposal. 1. Making mapping a mandatory step for usersFirst of all, I think making the mapping step mandatory for the user is a bad idea. Integrations like 2. Ability to provide non-continuous forecastsThis proposal suggests that the forecasts are supplied as single datapoints which contain a timestamp and a value. This model might work, but I would suggest that we should support passing some sort of Se we should support "2026-01-01T23:45:00": 10.0
"2026-01-02T00:00:00": "unknown"This example would mean that between 23:45-00:00 the price is 10.0, but from 00:00 on the price is unknown. With this we can also represent gaps in the forecast: "2026-01-01T23:45:00": 10.0
"2026-01-02T00:00:00": "unknown"
"2026-01-02T10:00:00": 20.0In this one the price is unknown between 00:00-10:00, but is known to be 3. Forecast caching / stateThis proposal does not really touch on how the forecast state and caching is handled. I suspect that we do not want to record the whole forecast into history every time it is updated like state attributes are now. This would be too expensive to do on all forecast entities and would be of dubious value anyways. I would argue that we should have the most recent forecast stored as a state or state attributes of some sort. This would then let use forecasts in the UI without having to worry about how fetching the forecast each time could have external effects. I assume that in this proposal each forecast integration decides whether they fetch they return cached values or fetch / calculate new ones every time. I think we should have a similar approach as we do with other entities where the integration can decide how often the forecast entity is updated and the results are then stored/cached in memory in a standard way. 4. relationship between sensor and forecast entity(ies)I am imagining from an UX perspective that when we open an entity more info panel in the UI, we should be able to see the history and the forecast in the same view. With this proposal we could have several forecast entities per sensor, which is good so we can support multiple different forecast models for example. However, it does introduce a UX question of how would we display the forecast with the main entity history. Maybe the solution is to show the history + forecast in the forecast entity only? Or maybe we should be able to pick the main forecast entity for a target entity? I do not know and this is probably outside the scope of this proposal. Still good to think about the implications. 5. Uncertainty for valuesForecasts can be and often are non deterministic. We should not implement the feature in a way that prevents us from implementing this in the future. For example we could have quantiles or something similar to represent the forecast confidence for certain bands of values instead of just offering a single point value. 6. How forecasts should be interpretedThe easiest way to deal with forecasts is to assume that they are just step functions. It would be possible to interpret forecasts as interpolated values between points, which would then allow us to ask for forecasts for very specific points in time. This might be nice, but I would argue that step functions are good enough if we just make sure that the forecasts have enough datapoints. |
|
Updated the proposal. A forecast is now represented as a regular entity with a 1:1 relationship to a target entity. Each forecast entity predicts a single target entity. For example, with Nordpool, there would be one forecast entity for the current price sensor. For future ML forecasting integrations, there could be multiple forecast entities for the same target, each representing a different ML model. Since the integration knows the relationship between the forecast and its target entity (due to setting up and specifying a source entity to learn from or so), it provides this mapping automatically. |
|
This is now being actively discussed within the core team. We first need to settle the debate between this forecast entity model and other alternatives like a forecast registry. If we conclude that the entity model is the way to go, I'll update here. |
|
I think Matter is an interesting reference here, especially the Device Energy Management cluster. Matter's "forecast" is not always just a datetime → value prediction. It can also describe a future schedule, with slots and constraints such as earliest/latest start time, min/max power, or whether the operation is pausable. I don't think HA should reproduce the Matter model, but I'd be careful not to make ForecastPoint(datetime, value) the fundamental abstraction. This would also make the Matter integration a natural future consumer/provider of the same Forecast API. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Context
Today, forecast-capable integrations in Home Assistant are fragmented:
This leads to duplicated logic, poor interoperability, and no consistent UX for forecasted values such as energy prices, energy production, energy usage, and similar time-based data.
A first proposal, #1357, suggested adding forecast support directly to
SensorEntity. That solves the case where the entity's own integration also owns the forecast data. The architecture discussion raised another use case: a forecast provider may be able to forecast input data from an arbitrary existing entity. A sensor-owned API is too rigid for that case.This proposal introduces forecast entities. A forecast entity represents one forecast for one target entity.
Proposal
Introduce a
forecastcomponent in core.Integrations expose forecast data by creating forecast entities through a
forecast.pyplatform.A forecast entity is separate from the target entity, but it has exactly one target entity:
The forecast entity declares which entity it forecasts through a stable target reference. The forecast component resolves that reference through the entity registry and exposes the current
target_entity_id.For example, Nord Pool can create one forecast entity for each current-price sensor. The user does not need to configure a mapping before forecasts work.
Multiple forecast entities may target the same entity. This supports advanced cases where different integrations or models can forecast the same target sensor.
Example:
For model-based or machine-learning integrations, the integration creates one forecast entity for each target entity it predicts. The model's source entities, recorder queries, training data, and feature inputs remain internal to that integration. Core only needs to know which target entity the forecast applies to and how to retrieve the forecast values.
For the first version, the forecast component should compute a primary forecast entity for each target entity. The frontend can use the primary forecast in more-info and history views.
Primary selection is deterministic:
This does not require a stored primary mapping in the first version. A future UI can allow users to choose a different primary forecast.
Forecast entity model
A forecast entity exposes:
async_forecast()The target entity is the entity whose future value is being forecast.
Integrations must not build or store the target entity's
entity_idas the durable relationship. Entity IDs can be changed by the user.Integrations should only provide a
ForecastTargetRef. The forecast component is responsible for resolving that reference to the current target entity registry entry and currententity_id.For integration-owned targets, the forecast entity can link to the target by stable integration data:
For user-selected or third-party targets, the integration can store the target entity registry entry ID selected during configuration:
The forecast component resolves either reference type and exposes
target_entity_idas a runtime property and state attribute. Integrations do not need to resolve the current entity ID themselves.The target entity reference and device relationship should be handled like helper sensors such as the derivative helper.
For the first version, valid targets are numeric
sensorentities.Forecast values use the target entity's native meaning and unit. A forecast for
sensor.nord_pool_se3_current_pricereturns future values in that sensor's native unit. If a provider cannot produce values compatible with the target entity, it should not create a forecast entity for that target.Forecast entity state
The forecast entity state contains the timestamp when the provider last updated its cached forecast data.
The forecast payload must not be stored in the entity state or in state attributes. Forecast values are only exposed through
forecast.get_forecastsandforecast/get_forecasts.If no forecast has been loaded yet, the state is
unknown.If the provider cannot currently operate, the entity uses the normal Home Assistant
unavailableavailability model.The state should not be the current forecast value, the next forecast value, or any other forecast datapoint.
Forecast entity state attributes
Forecast entity attributes should contain only small metadata needed to understand the relationship and status of the forecast entity.
Required attribute:
target_entity_id: the current entity ID of the entity whose future value is forecast, resolved from the stable target referenceOptional attributes:
forecast_type: a short provider-defined type, for exampleprice,production,consumption, ordemandforecast_source: a short provider-defined source or model namenative_unit_of_measurement: the native unit used by forecast values, normally matching the target entityfirst_forecast_datetime: first datetime available in the current cached forecast, if knownlast_forecast_datetime: last datetime available in the current cached forecast, if knownThe following must not be state attributes:
Frontend and automation consumers should use
forecast.get_forecastsorforecast/get_forecaststo retrieve forecast data instead of reading forecast values from attributes.Example state:
Target lifecycle
A forecast entity must link to its target through a stable registry reference, not by storing the target entity ID.
The forecast component owns target resolution and lifecycle handling for forecast entities.
If the target entity is renamed, the forecast entity continues to work and exposes the current resolved
target_entity_id.If the target entity is temporarily unavailable, the forecast entity follows the provider's normal availability. If the provider depends on the target entity's current state or history and cannot produce a forecast while that data is unavailable, the forecast entity should become unavailable. If the provider can still produce a forecast from its own upstream data, model inputs, or cached forecast data, the forecast entity may remain available. This rule is the same for integration-owned and third-party forecast entities.
The forecast component only needs to resolve and validate the target registry entry. It does not need to interpret the target entity's runtime state when deciding forecast availability.
If the target entity registry entry no longer exists, the forecast entity should become unavailable and should not return forecasts.
If the target entity is disabled by the user, the forecast entity should not appear as an active forecast for that target and discovery should not return it for normal UI use. The forecast entity should become unavailable until the target is enabled again, or until the forecast entity is reconfigured or removed. This rule is the same for integration-owned and third-party forecast entities.
If the forecast entity and target entity are created by the same config entry, unloading or removing that config entry unloads or removes both entities together.
If a third-party forecast integration targets an entity from another config entry, removing the target should leave the forecast entity unavailable until the user reconfigures or removes it.
Forecast retrieval should return no forecast for a forecast entity whose target registry entry is missing or disabled.
Discovery
Home Assistant can discover forecasts for a target entity by finding forecast entities whose
target_entity_idmatches the target.Internally, discovery should compare the resolved target registry entry. The
entity_idis the user-facing lookup value, not the durable relationship between a forecast entity and its target.A websocket API can expose this:
Request:
Response:
For third-party forecasting integrations, the integration creates forecast entities for the targets selected during its own configuration flow.
Forecast retrieval
Forecast data is retrieved through a shared API:
forecast.get_forecastsforecast/get_forecastsThe request can target either:
The request can also include an optional timeframe:
start_datetimeend_datetimestart_datetimeandend_datetimecan be specified independently. A request may include either bound, both bounds, or neither.start_datetimeis provided, providers return forecasts from that time onward, up to their default forecast end.end_datetimeis provided, providers return forecasts up to that time, including the point needed to represent the value active atnow.For step-series forecasts, a timeframe query should include enough context to interpret the value at
start_datetime. If the value active atstart_datetimestarted before the requested window, the response should include that preceding forecast point. This avoids forcing consumers to make a second request to understand the first interval.The response should not return past datapoints other than the most recent point needed to represent the value active at the start of the returned window. If no
start_datetimeis provided, this means the most recent point needed to represent the value active atnow.Example targeting the sensor:
Example response:
Forecast data model
async_forecast()returns:list[ForecastPoint]NoneNonemeans no forecast is currently available.Each
ForecastPointcontains:datetimevalueThe first version uses step-series semantics:
datetimeuntil the next forecast pointvalue: nullmeans the value is explicitly unknown from that timestamp until the next forecast pointvalue: nullpoint unless the provider has a valid reason for the final value to remain valid beyond the returned forecast windowvalue: null, consumers treat the final value as valid until the provider refreshes or replaces the forecastThis allows integrations to represent known end boundaries and gaps without overlapping intervals.
Example:
This means the value is
10.0from 23:45 until 00:00, unknown from 00:00 until 10:00,20.0from 10:00 until 11:00, and unknown after 11:00.Core should validate and normalize forecast output:
nullis allowed only to represent an explicitly unknown forecast valuenullpoint to mark where the known forecast endsCaching and state history
Forecast payloads should not be stored in state attributes and should not be recorded in history.
Forecast entities may cache the latest forecast in memory. The integration controls refresh timing, similar to the weather forecast model. Calling
forecast.get_forecastsshould not require every provider to perform a fresh external API call.The forecast entity state and small metadata attributes may be recorded like normal entity state. This records forecast refresh timing and metadata changes, not forecast values or the full forecast payload.
Forecast entity states should not be included in long-term statistics in the first version. Statistics for forecasted values, forecast accuracy, forecast error, and forecast-vs-actual comparisons are separate future features and should not be implied by
state_class.Entity updates
Forecast entities update like regular Home Assistant entities.
An integration can update a forecast entity by using the same patterns it already uses for other entities:
async_update()DataUpdateCoordinatorWhen the integration refreshes forecast data, it updates its in-memory forecast cache, sets the entity state to the forecast update timestamp, updates the small metadata attributes, and writes the entity state.
The forecast entity does not need to update when the current forecast interval changes unless the provider refreshed or replaced the cached forecast payload.
Calling
forecast.get_forecastsreads from the provider's current forecast data. It should not be the primary mechanism that updates entity state. If a provider needs fresh data before it can answer, the integration may refresh its own cache, but the regular entity update path remains responsible for keeping state and attributes current.This keeps forecast entities consistent with the rest of Home Assistant:
Consequences
This creates one shared forecast API for frontend, scripts, automations, and future dashboards.
The model supports both simple and advanced cases:
The model avoids a separate mapping store for the first version. The relationship between a forecast and its target is part of the forecast entity.
The initial target scope is intentionally narrow. Limiting the first version to numeric sensors keeps the schema and UI use cases clear while still covering immediate energy-price, energy-usage, energy-production, and similar integrations.
Implementation outline
Add a
forecastcomponent.The component owns forecast entity registration, target reference resolution, target validation, discovery indexing, target lifecycle handling, and the shared action/websocket API.
Add
ForecastEntity.Forecast entities expose a stable target reference, expose a state containing the forecast update timestamp, expose small metadata attributes, and implement
async_forecast(). The forecast component resolvestarget_entity_idfrom the target reference.Add forecast discovery.
Implement
forecast/get_forecast_entitiesso the frontend can find forecast entities for a target entity. Discovery should not return forecast entities whose target registry entry is missing or disabled for normal UI use. The response should include the computed primary forecast entity ID.Add forecast retrieval.
Implement
forecast.get_forecastsandforecast/get_forecasts, returning forecasts keyed by target entity and forecast entity. The request should support optionalstart_datetimeandend_datetimefilters. The response should mark which forecast entity is primary for each target.Add forecast validation and normalization.
Validate datetimes, sorting, duplicate points, numeric values, and explicit unknown values.
Add frontend support.
The frontend can automatically use the computed primary forecast entity for the target entity's more-info/history view.
Add an initial integration.
Start with an integration-owned forecast entity such as Nord Pool to prove target linkage, state updates, caching, step-series semantics, and the frontend path.
Cover target lifecycle behavior.
Add tests for renamed targets, target device association changes, provider-dependent and provider-independent temporarily unavailable targets, disabled targets, removed target registry entries, same-config-entry unload, and third-party target removal.
Out of scope for the first version
These are valid future use cases, but the first version should establish the forecast entity model, target linkage, state model, retrieval API, caching behavior, and forecast time semantics first.
Integrations that benefit
Immediate candidates:
nordpoolamberelectricenergyzeroeasyenergytibberpvpc_hourly_pricingHistory
2026-07-21: Revised draft to be more precise and make the forecast a regular entity with a 1:1 relationship to the target entity
2026-07-21: add null termination, make primary forecast entity predictable
2026-07-21: Be more precise on the
start_datetimeandend_datetimefilterAll reactions