Add Home Load Support

CHANGELOG.md

@@ -5,9 +5,12 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
 ## [0.1.0-rev3]
 
 ### Added
+
 - **Mining Performance Analysis Domain** (`edge_mining/domain/performance/`):
   - Value objects: `MiningReward`, `PoolWorkerStats`, `PoolStats`, `PayoutSchedule` in `value_objects.py` (renamed from misspelled `values_objects.py`)
   - Entity `MiningSession` for tracking aggregated mining activity (`entities.py`)
@@ -61,12 +64,49 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Tests** — 107 new unit tests across tracker adapters, configuration service, REST router, and CLI commands
 - **Tests** — 15 new unit tests for `CachedRateLimitedTrackerBase` (cache hit/miss, TTL expiry, backoff progression, stale-while-error, retry-after handling, cache invalidation) plus 429-detection tests for the Ocean and Braiins adapters
 
+- **Home Load — Phase 3: DecisionalContext Integration**
+  - Extended field resolver (`helpers.py`) with dict key lookup for `home_load.devices.<name>.*` paths and `None` guard for `Optional` intermediate fields
+  - Pre-computed window properties on `LoadEnergyConsumption`: `next_1h`, `next_2h`, `next_4h`, `last_1h`, `last_4h`, `last_24h`
+  - Example YAML rules: `home_load_start_rules.yaml` (3 rules), `home_load_stop_rules.yaml` (4 rules)
+  - Fixed existing rules from `home_load_forecast` → `home_load.total_forecast.next_2h.avg_power`
+  - 5 new unit tests for dict resolver
+
+- **Home Load — Phase 4: ML Forecast Providers (Statsmodels + XGBoost)**
+  - ML optional dependencies: `scikit-learn>=1.5.0`, `statsmodels>=0.14.0`, `xgboost>=2.0.0` in `[ml]` extras
+  - `EnergyLoadForecastProviderAdapter.STATSMODELS` and `.XGBOOST` enum values
+  - Config dataclasses: `EnergyLoadForecastProviderStatsmodelsConfig`, `EnergyLoadForecastProviderXGBoostConfig`
+  - Feature engineering utilities (`features.py`): `intervals_to_hourly_series()`, `fill_missing_hours()`, `build_calendar_features()`, `build_lag_features()`, `prepare_supervised_dataset()`
+  - `LoadConsumptionModel` entity with `model_bytes` (serialized pickle), MAE/RMSE metrics, `is_active` flag
+  - `LoadConsumptionModelRepository` port + three implementations: InMemory, SQLite, SQLAlchemy
+  - `load_consumption_models` database table with composite index on `(adapter_type, device_id, is_active)`
+  - Alembic migration `c3d4e5f6a7b8` for `load_consumption_models` table
+  - `StatsmodelsForecastProvider` (Holt-Winters exponential smoothing) with factory, lazy import
+  - `XGBoostForecastProvider` (gradient boosting with calendar + lag features) with factory, lazy import
+  - `LoadForecastModelTrainingService`: nightly batch training with holdout evaluation + best model promotion
+  - Pydantic schemas: `EnergyLoadForecastProviderStatsmodelsConfigSchema`, `EnergyLoadForecastProviderXGBoostConfigSchema`
+  - Scheduler cron job at 04:00 for nightly ML model training
+
+- **Home Load — API Completion**
+  - `ConfigurationServiceInterface`: 10 new abstract CRUD methods for `EnergyLoadForecastProvider` (5) and `EnergyLoadHistoryProvider` (5)
+  - `ConfigurationService`: implemented `add_`, `get_`, `list_`, `update_`, `remove_energy_load_forecast_provider`
+  - Completed 5 forecast provider REST endpoints (previously stubs returning 501/empty):
+    - `GET /energy-load-forecast-providers` — list all providers
+    - `POST /energy-load-forecast-providers` — create and persist provider
+    - `GET /energy-load-forecast-providers/{id}` — get provider by ID
+    - `PUT /energy-load-forecast-providers/{id}` — update provider with config deserialization
+    - `DELETE /energy-load-forecast-providers/{id}` — remove provider
+
 ### Changed
 - `MiningPerformanceTrackerPort` methods are now `async`; the dummy tracker adapter has been adapted accordingly
 - `OptimizationService` now awaits `get_current_hashrate` calls to match the async port contract, and consolidates the three tracker calls behind a new private helper `_build_mining_performance_snapshot` that returns a single `MiningPerformanceSnapshot`
 - Replaced `DecisionalContext.tracker_current_hashrate: Optional[HashRate]` with `mining_performance: Optional[MiningPerformanceSnapshot]`; `DecisionalContextSchema` and the rule engine `OPERATOR_EXAMPLES[LTE]` example updated accordingly (new field path: `mining_performance.current_hashrate.value`)
 - Interactive CLI main menu: "Run all optimization units" shifted from option 8 to 9 to accommodate the new tracker menu at option 8
 - Replaced per-module `_utc_now_timestamp()` helpers in `domain/performance/entities.py` and `domain/performance/value_objects.py` with the shared `utc_now_timestamp()` from `domain/common.py`
+- `AdapterService`: new factory branches for STATSMODELS and XGBOOST with `model_repo` injection
+- `PersistenceSettings`: added `load_consumption_model_repo` field
+- `Services` dataclass: added `load_forecast_training_service` field
+- `AutomationScheduler`: accepts optional `load_forecast_training_service`, schedules nightly training
+- `bootstrap.py`: `LoadConsumptionModelRepository` wired in all three persistence branches (InMemory/SQLite/SQLAlchemy)
 
 ### Fixed
 - Replaced latent `default_factory=Timestamp(datetime.now())` bugs (which froze a single timestamp at class-definition time) with the proper callable `utc_now_timestamp`, producing a fresh timestamp per instance

alembic.ini

@@ -86,7 +86,7 @@ path_separator = os
 # database URL.  This is consumed by the user-maintained env.py script only.
 # other means of configuring database URLs may be customized within the env.py
 # file.
-sqlalchemy.url = sqlite:///./edgemining.db
+sqlalchemy.url = sqlite:///data/db/edgemining.db
 
 
 [post_write_hooks]

alembic/versions/4e55fe6113c7_initial_schema_with_all_tables.py

@@ -98,12 +98,12 @@ def upgrade() -> None:
     )
     op.create_index(op.f("ix_forecast_providers_id"), "forecast_providers", ["id"], unique=False)
     op.create_table(
-        "home_forecast_providers",
+        "energy_load_forecast_providers",
         sa.Column("id", sa.String(), nullable=False),
         sa.Column("name", sa.String(), nullable=False),
         sa.Column("adapter_type", sa.String(), nullable=False),
         sa.Column(
-            "config", edge_mining.adapters.domain.home_load.tables.HomeForecastProviderConfigType(), nullable=True
+            "config", edge_mining.adapters.domain.home_load.tables.EnergyLoadForecastProviderConfigType(), nullable=True
         ),
         sa.Column("external_service_id", sa.String(), nullable=True),
         sa.ForeignKeyConstraint(
@@ -112,7 +112,9 @@ def upgrade() -> None:
         ),
         sa.PrimaryKeyConstraint("id"),
     )
-    op.create_index(op.f("ix_home_forecast_providers_id"), "home_forecast_providers", ["id"], unique=False)
+    op.create_index(
+        op.f("ix_energy_load_forecast_providers_id"), "energy_load_forecast_providers", ["id"], unique=False
+    )
     op.create_table(
         "miner_controllers",
         sa.Column("id", sa.String(), nullable=False),
@@ -206,7 +208,6 @@ def upgrade() -> None:
             "target_miner_ids", edge_mining.adapters.domain.optimization_unit.tables.EntityIdListType(), nullable=False
         ),
         sa.Column("energy_source_id", sa.String(), nullable=True),
-        sa.Column("home_forecast_provider_id", sa.String(), nullable=True),
         sa.Column("performance_tracker_id", sa.String(), nullable=True),
         sa.Column(
             "notifier_ids", edge_mining.adapters.domain.optimization_unit.tables.EntityIdListType(), nullable=False
@@ -215,23 +216,34 @@ def upgrade() -> None:
             ["energy_source_id"],
             ["energy_sources.id"],
         ),
-        sa.ForeignKeyConstraint(
-            ["home_forecast_provider_id"],
-            ["home_forecast_providers.id"],
-        ),
         sa.ForeignKeyConstraint(
             ["performance_tracker_id"],
             ["mining_performance_trackers.id"],
         ),
         sa.PrimaryKeyConstraint("id"),
     )
     op.create_index(op.f("ix_optimization_units_id"), "optimization_units", ["id"], unique=False)
+    op.create_table(
+        "home_load_power_points",
+        sa.Column("device_id", sa.String(), nullable=False),
+        sa.Column("timestamp", sa.DateTime(timezone=True), nullable=False),
+        sa.Column("power", sa.Float(), nullable=False),
+        sa.PrimaryKeyConstraint("device_id", "timestamp"),
+    )
+    op.create_index(
+        "ix_home_load_power_points_device_ts",
+        "home_load_power_points",
+        ["device_id", "timestamp"],
+        unique=False,
+    )
     # ### end Alembic commands ###
 
 
 def downgrade() -> None:
     """Downgrade schema."""
     # ### commands auto generated by Alembic - please adjust! ###
+    op.drop_index("ix_home_load_power_points_device_ts", table_name="home_load_power_points")
+    op.drop_table("home_load_power_points")
     op.drop_index(op.f("ix_optimization_units_id"), table_name="optimization_units")
     op.drop_table("optimization_units")
     op.drop_index(op.f("ix_miners_id"), table_name="miners")
@@ -244,8 +256,8 @@ def downgrade() -> None:
     op.drop_table("mining_performance_trackers")
     op.drop_index(op.f("ix_miner_controllers_id"), table_name="miner_controllers")
     op.drop_table("miner_controllers")
-    op.drop_index(op.f("ix_home_forecast_providers_id"), table_name="home_forecast_providers")
-    op.drop_table("home_forecast_providers")
+    op.drop_index(op.f("ix_energy_load_forecast_providers_id"), table_name="energy_load_forecast_providers")
+    op.drop_table("energy_load_forecast_providers")
     op.drop_index(op.f("ix_forecast_providers_id"), table_name="forecast_providers")
     op.drop_table("forecast_providers")
     op.drop_index(op.f("ix_energy_monitors_id"), table_name="energy_monitors")

alembic/versions/b2c3d4e5f6a7_add_energy_load_history_providers.py

@@ -0,0 +1,56 @@
+"""Add energy_load_history_providers table
+
+Revision ID: b2c3d4e5f6a7
+Revises: a1b2c3d4e5f6
+Create Date: 2026-04-22 10:00:00.000000
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+
+import edge_mining.adapters.domain.home_load.tables
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "b2c3d4e5f6a7"
+down_revision: Union[str, Sequence[str], None] = "a1b2c3d4e5f6"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    """Add energy_load_history_providers table."""
+    op.create_table(
+        "energy_load_history_providers",
+        sa.Column("id", sa.String(), nullable=False),
+        sa.Column("name", sa.String(), nullable=False),
+        sa.Column("adapter_type", sa.String(), nullable=False),
+        sa.Column(
+            "config",
+            edge_mining.adapters.domain.home_load.tables.EnergyLoadHistoryProviderConfigType(),
+            nullable=True,
+        ),
+        sa.Column("external_service_id", sa.String(), nullable=True),
+        sa.ForeignKeyConstraint(
+            ["external_service_id"],
+            ["external_services.id"],
+        ),
+        sa.PrimaryKeyConstraint("id"),
+    )
+    op.create_index(
+        op.f("ix_energy_load_history_providers_id"),
+        "energy_load_history_providers",
+        ["id"],
+        unique=False,
+    )
+
+
+def downgrade() -> None:
+    """Remove energy_load_history_providers table."""
+    op.drop_index(
+        op.f("ix_energy_load_history_providers_id"),
+        table_name="energy_load_history_providers",
+    )
+    op.drop_table("energy_load_history_providers")

alembic/versions/c3d4e5f6a7b8_add_load_consumption_models.py

@@ -0,0 +1,47 @@
+"""Add load_consumption_models table
+
+Revision ID: c3d4e5f6a7b8
+Revises: b2c3d4e5f6a7
+Create Date: 2026-04-22 14:00:00.000000
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "c3d4e5f6a7b8"
+down_revision: Union[str, None] = "b2c3d4e5f6a7"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "load_consumption_models",
+        sa.Column("id", sa.String(), nullable=False),
+        sa.Column("device_id", sa.String(), nullable=True),
+        sa.Column("adapter_type", sa.String(), nullable=False),
+        sa.Column("trained_at", sa.DateTime(timezone=True), nullable=True),
+        sa.Column("mae", sa.Float(), nullable=True),
+        sa.Column("rmse", sa.Float(), nullable=True),
+        sa.Column("samples_used", sa.Integer(), nullable=False, server_default="0"),
+        sa.Column("is_active", sa.Boolean(), nullable=False, server_default="0"),
+        sa.Column("model_bytes", sa.LargeBinary(), nullable=True),
+        sa.PrimaryKeyConstraint("id"),
+    )
+    op.create_index("ix_load_consumption_models_id", "load_consumption_models", ["id"])
+    op.create_index(
+        "ix_load_consumption_models_active",
+        "load_consumption_models",
+        ["adapter_type", "device_id", "is_active"],
+    )
+
+
+def downgrade() -> None:
+    op.drop_index("ix_load_consumption_models_active", table_name="load_consumption_models")
+    op.drop_index("ix_load_consumption_models_id", table_name="load_consumption_models")
+    op.drop_table("load_consumption_models")

alembic/versions/d4e5f6a7b8c9_add_tuning_and_backtesting_columns.py

@@ -0,0 +1,35 @@
+"""Add tuning and backtesting columns to load_consumption_models
+
+Revision ID: d4e5f6a7b8c9
+Revises: c3d4e5f6a7b8
+Create Date: 2026-04-25 10:00:00.000000
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "d4e5f6a7b8c9"
+down_revision: Union[str, None] = "c3d4e5f6a7b8"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    with op.batch_alter_table("load_consumption_models") as batch_op:
+        batch_op.add_column(sa.Column("tuning_params", sa.Text(), nullable=True))
+        batch_op.add_column(sa.Column("backtest_mae", sa.Float(), nullable=True))
+        batch_op.add_column(sa.Column("backtest_rmse", sa.Float(), nullable=True))
+        batch_op.add_column(sa.Column("backtest_folds", sa.Integer(), nullable=False, server_default="0"))
+
+
+def downgrade() -> None:
+    with op.batch_alter_table("load_consumption_models") as batch_op:
+        batch_op.drop_column("backtest_folds")
+        batch_op.drop_column("backtest_rmse")
+        batch_op.drop_column("backtest_mae")
+        batch_op.drop_column("tuning_params")

alembic/versions/e5f6a7b8c9d0_add_home_loads_profile_to_optimization_units.py

@@ -0,0 +1,29 @@
+"""Add home_loads_profile to optimization_units
+
+Revision ID: e5f6a7b8c9d0
+Revises: d4e5f6a7b8c9
+Create Date: 2026-04-29 12:00:00.000000
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "e5f6a7b8c9d0"
+down_revision: Union[str, None] = "d4e5f6a7b8c9"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    with op.batch_alter_table("optimization_units") as batch_op:
+        batch_op.add_column(sa.Column("home_loads_profile", sa.String(), nullable=True))
+
+
+def downgrade() -> None:
+    with op.batch_alter_table("optimization_units") as batch_op:
+        batch_op.drop_column("home_loads_profile")

data/examples/policies/550e8400-e29b-41d4-a716-446655440000.yaml

@@ -119,7 +119,7 @@ stop_rules:
     description: Stop mining during high home energy demand periods
     conditions:
       all_of:
-        - field: home_load_forecast
+        - field: home_load.total_forecast.next_2h.avg_power
           operator: gt
           value: 2800
         - field: timestamp.hour
@@ -136,6 +136,6 @@ stop_rules:
     enabled: true
 metadata:
   author: Edge Mining User
-  version: 5
+  version: 6
   created: '2025-08-04'
-  last_modified: '2026-01-18'
+  last_modified: '2026-04-22'

data/examples/rules/start/advanced_start_rules.yaml

@@ -38,9 +38,9 @@ rules:
         - field: "energy_state.battery.state_of_charge"
           operator: "gt"
           value: 60
-        - field: "home_load_forecast"
+        - field: "home_load.total_forecast.next_2h.avg_power"
           operator: "lt"
-          value: 2000  # Low expected home consumption
+          value: 2000  # Low expected home consumption (next 2h average)
         - any_of:
             - field: "energy_state.production"
               operator: "gt"