Skip to content

Fix Forecast.Solar timezone misalignment in non-UTC timezones#2856

Closed
Copilot wants to merge 3 commits into
mainfrom
copilot/fix-forecast-solar-timezone-issue
Closed

Fix Forecast.Solar timezone misalignment in non-UTC timezones#2856
Copilot wants to merge 3 commits into
mainfrom
copilot/fix-forecast-solar-timezone-issue

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Nov 2, 2025

Forecast.Solar data displayed at incorrect local times in non-UTC timezones, showing solar production during nighttime hours (e.g., 17:00-01:00 in Adelaide UTC+10:30).

Root Cause

SolarAPI.fetch_pv_forecast() calculated midnight_utc as UTC midnight instead of using the user's local midnight from base.midnight_utc. This caused forecast timestamps to be offset by the timezone difference when computing minute offsets.

Changes

apps/predbat/solcast.py:863-866

  • Use base.now_utc and base.midnight_utc instead of creating new datetime objects
  • Fix timedelta calculation: .total_seconds().seconds for timezone-aware datetime differences
# Before
self.now_utc = datetime.now(timezone.utc)
self.midnight_utc = self.now_utc.replace(hour=0, minute=0, second=0, microsecond=0)
self.minutes_now = int((self.now_utc - self.midnight_utc).seconds / 60 / PREDICT_STEP) * PREDICT_STEP

# After
self.now_utc = self.base.now_utc
self.midnight_utc = self.base.midnight_utc
self.minutes_now = int((self.now_utc - self.midnight_utc).total_seconds() / 60 / PREDICT_STEP) * PREDICT_STEP

Forecast.Solar returns timestamps in location's local timezone. Line 289 converts to UTC correctly, but minute offset calculation (line 298) required user's local midnight as reference, not UTC midnight.

Original prompt

This section details on the original issue you should resolve

<issue_title>Forecast.Solar Timezone issue</issue_title>
<issue_description>Describe the bug
Using the forecast solar method in apps.yaml PV production time alignment does not match the local timezone or solar day for lat/long

Expected behaviour
Given this input, I expected the solar forecast to align to the local Adelaide time ACDT, which the planning portion does, but is showing PV production for nighttime hours

pred_bat:
  module: predbat
  class: PredBat

  prefix: predbat
  timezone: Australia/Adelaide
  run_every: 5

  #ha_url: "http://homeassistant"
  #ha_key: "secret_ha_key.replaceme"

  # Replace this with the total PV energy generated today from your inverter
  pv_today:
    - sensor.my_home_solar_generated

  # Currency, symbol for main currency second symbol for 1/100s e.g. $ c or £ p or e c
  currency_symbols:
    - "$"
    - "c"

  # Misc
  charge_control_immediate: False
  num_cars: 0 # I don't have an EV :(

  inverter_type: TESLA
  inverter:
    name: "Tesla Powerwall via Teslemetry"
    has_rest_api: False
    has_mqtt_api: False
    output_charge_control: "none"
    has_charge_enable_time: False
    has_discharge_enable_time: False
    has_target_soc: False
    # While the Powerwall does have a reserve SoC, we don't need to
    # leverage it for controlling charge/discharge
    has_reserve_soc: False
    charge_time_format: "S"
    charge_time_entity_is_option: False
    soc_units: "%"
    num_load_entities: 1
    time_button_press: False
    clock_time_format: "%Y-%m-%d %H:%M:%S"
    write_and_poll_sleep: 2
    has_time_window: False
    support_charge_freeze: False
    support_discharge_freeze: False
    has_idle_time: False

  # ---- Live power ----
  battery_power:
    - sensor.my_home_battery_power
  battery_power_invert:
    - False

  pv_power:
    - sensor.my_home_solar_power

  load_power:
    - sensor.my_home_load_power

  grid_power:
    - sensor.my_home_grid_power

  grid_power_invert:
    - True

  inverter_reserve_max: 80 #Anything between 80-100 will always be treated as 100

  # ---- Daily energy (kWh, cumulative today) ----
  load_today:
    - sensor.my_home_home_usage
  import_today:
    - sensor.my_home_grid_imported
  export_today:
    - sensor.my_home_grid_exported

  # ---- State of charge ----
  soc_percent:
    - sensor.my_home_percentage_charged
  soc_max:
    - "27" # ensure this matches your usable kWh

  # ---- Powerwall controls via Teslemetry (must be writable) ----
  allow_charge_from_grid:
    - switch.my_home_allow_charging_from_grid
  allow_export:
    - select.my_home_allow_export

  # ---- Solar forecast (Solcast) ----
  #pv_forecast_today: sensor.energy_production_today
  #pv_forecast_tomorrow: sensor.energy_production_tomorrow

  forecast_solar:
   - latitude: -34.0
     longitude: 138.0
     kwp: 4.3
     azimuth: 6
     declination: 23
     efficiency: 0.95
  forecast_solar_max_age: 4

  # ---- Tariff sensors (Octopus) ----
  metric_octopus_import: "re:(sensor.(octopus_energy_|)electricity_[0-9a-z]+_[0-9a-z]+_current_rate)"
  metric_octopus_export: "re:(sensor.(octopus_energy_|)electricity_[0-9a-z]+_[0-9a-z]+_export_current_rate)"
  metric_standing_charge: "re:(sensor.(octopus_energy_|)electricity_[0-9a-z]+_[0-9a-z]+_current_standing_charge)"
  octopus_free_session: "re:(event.octopus_energy_([0-9a-z_]+|)_octoplus_free_electricity_session_events)"
  threads: auto
  forecast_hours: 48

  # --- Predbat service hooks (Tesla / Teslemetry) ---
  # These hooks are called when Predbat wants to change the current
  # state of charge/discharge. They can tie-in to other HA entities.
  #
  # Tesla PW Operation modes can be one of:
  # ['self_consumption','autonomous','backup']
  #
  # grid-charging=on, mode=backup: Powerwall will charge
  # grid-charging=off, mode=backup: Powerwall will hold
  # mode=self_consumption: Powerwall will discharge

  charge_start_service:
    - service: switch.turn_on
      entity_id: switch.my_home_allow_charging_from_grid
      repeat: True
    - service: select.select_option
      entity_id: select.my_home_operation_mode
      option: "backup"
      repeat: True

  charge_hold_service:
    - service: switch.turn_off
      entity_id: switch.my_home_allow_charging_from_grid
      repeat: True
    - service: select.select_option
      entity_id: select.my_home_operation_mode
      option: "backup"
      repeat: True

  charge_freeze_service:
    - service: switch.turn_off
      entity_id: switch.my_home_allow_charging_from_grid
      repeat: True
    - service: select.select_option
      entity_id: select.my_home_operation_mode
      option: "backup"
      repeat: True

  charge_stop_service:
    - service: switch.turn_off
      entity_id: switch.my_home_allow_charging_from_grid
    - service: select.select_option
      entity_id: select.my_home_operation_mode
      option: "self_consumption"

Predbat version

Screenshots
If appl...


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits November 2, 2025 21:20
Co-authored-by: springfall2008 <48591903+springfall2008@users.noreply.github.com>
Co-authored-by: springfall2008 <48591903+springfall2008@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix forecast solar method timezone alignment Fix Forecast.Solar timezone misalignment in non-UTC timezones Nov 2, 2025
Copilot AI requested a review from springfall2008 November 2, 2025 21:26
@springfall2008
Copy link
Copy Markdown
Owner

On main now

@springfall2008 springfall2008 deleted the copilot/fix-forecast-solar-timezone-issue branch May 4, 2026 11:07
tieskuh added a commit to tieskuh/batpred that referenced this pull request May 30, 2026
…mezones

## Summary

`solcast.py` line 508 incorrectly applies `.replace(tzinfo=pytz.utc)` to `self.midnight_utc`, which is already a timezone-aware datetime in `local_tz` (not UTC, despite the misleading name). This produces a final `period_start_stamp` that is shifted forward by the local UTC offset. For users east of UTC (CET +1, CEST +2, Adelaide +10:30, Auckland +13), Forecast.solar data appears later in the plan than it should — and west of UTC, earlier.

This is the root cause of the symptoms reported in springfall2008#2821 (Australia/Adelaide, +10:30) and springfall2008#2911 (Pacific/Auckland, +13) and is still reproducible in v8.39.7 in Europe/Amsterdam (+2 CEST) where the smaller offset makes the bug look like "the PV tail extends past sunset" rather than complete day inversion.

## Root cause

In `predbat.py:738-747`:

```python
self.now_utc_real = datetime.now(self.local_tz)         # tz-aware in local_tz
now_utc = self.now_utc_real + timedelta(minutes=skew)   # still tz-aware in local_tz
...
self.now_utc = now_utc                                  # name says UTC, value is local_tz
self.midnight_utc = now_utc.replace(hour=0, ...)        # local midnight in local_tz
```

So `self.midnight_utc` is the start of the local day, *as a timezone-aware datetime in local_tz* — not UTC midnight. The name is misleading but the value is correct for the way it's used elsewhere (e.g. `(period_end_stamp - self.midnight_utc)` works correctly because Python normalizes tz-aware subtraction to absolute time).

The bug at `solcast.py:508`:

```python
period_start_stamp = self.midnight_utc.replace(tzinfo=pytz.utc) + timedelta(minutes=minute)
```

`.replace(tzinfo=...)` on a tz-aware datetime **keeps the wall-clock time and swaps the label**. So `00:00 CEST` becomes `00:00 UTC` — a different absolute moment, 2 hours later. Every subsequent `period_start_stamp` is shifted by that local UTC offset.

The other use of `.replace(tzinfo=pytz.utc)` on line 376 (Open-Meteo path) is correct because the stamp there is genuinely naive (from `strptime(ts, "%Y-%m-%dT%H:%M")` without `%z`). The bug only affects the Forecast.solar code path.

## Reproduction (Europe/Amsterdam, 30 May 2026)

Forecast.solar API returns sunset at `"2026-05-30T19:44:12+00:00"` = 21:44 CEST. With current code:

```
midnight_utc:               2026-05-30 00:00:00+02:00  (local midnight, in CEST)
sunset stamp:               2026-05-30 19:44:12+00:00  (UTC from cache)
minutes_end:                1304.2 min                 (21h44m since 00:00 CEST)
period_start_stamp (BUG):   2026-05-30 21:44:00+00:00  (00:00 + 1304 min, but tz forced to UTC)
                            -> displayed as 23:44 CEST in plan  ← WRONG, sunset is 21:44
```

User-visible: Predbat plan shows non-zero PV until 23:00-23:30 local, ~2 hours after astronomical sunset. Same data via the Home Assistant Forecast.solar integration sensor (`sensor.energy_production_today`) renders correctly.

## Fix

Drop the `.replace(tzinfo=pytz.utc)`. `self.midnight_utc` is already tz-aware in `local_tz`; adding a `timedelta` preserves the timezone:

```diff
--- a/apps/predbat/solcast.py
+++ b/apps/predbat/solcast.py
@@ -505,7 +505,7 @@ class SolarAPI(ComponentBase):
                 for offset in range(0, self.plan_interval_minutes, 1):
                     pv50 += dp4(forecast_watt_data.get(minute + offset, 0) / 1000.0)
                 pv50 /= 60
-                period_start_stamp = self.midnight_utc.replace(tzinfo=pytz.utc) + timedelta(minutes=minute)
+                period_start_stamp = self.midnight_utc + timedelta(minutes=minute)
                 data_item = {"period_start": period_start_stamp.strftime(TIME_FORMAT), "pv_estimate": pv50}
                 if period_start_stamp in period_data:
                     period_data[period_start_stamp]["pv_estimate"] += pv50
```

After fix:

```
period_start_stamp (FIX):   2026-05-30 21:44:00+02:00  (00:00 + 1304 min in CEST)
                            -> displayed as 21:44 CEST in plan  ← CORRECT
```

## Regression check

For users in UTC the fix is a no-op. Both old and new code produce `2026-05-30 19:44:00+00:00` for UTC midnight + 1184 min. Tested with the same reproduction script using `tzinfo=pytz.utc` for `midnight_utc`. Verified delta = 0 seconds.

## Related issues

- springfall2008#2821 — Australia/Adelaide, +10:30 offset, plan showed PV during night hours
- springfall2008#2911 — Pacific/Auckland, +13 offset, plan entirely shifted by 13h

Both were partially addressed by adding `?time=utc` to the URL and the `strptime` change in springfall2008#2856 / springfall2008#2911, but the underlying `.replace(tzinfo=pytz.utc)` on the local-tz `midnight_utc` was kept and is the remaining root cause.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Forecast.Solar Timezone issue

2 participants