feat: publish marginal cost matrix to predbat gateway by mgazza · Pull Request #3783 · springfall2008/batpred

mgazza · 2026-04-12T08:29:27Z

Summary

Extends the predbat_data MQTT payload sent to the ESP32 gateway device (GatewayMQTT._publish_predbat_data) with the marginal cost matrix produced by the Marginal mixin (apps/predbat/marginal.py).

The gateway's range display uses this to decide the colour of its appliance RAG indicators (oven, dryer, wash, EV). Previously the firmware had to infer cost from slot categories, which was misleading — e.g. a SOLAR slot with 100W forecast PV vs 3kW would both look GREEN for the dryer. Sending the actual marginal matrix lets the firmware show the real cost impact.

Payload additions

{
  "marginal_costs": [
    [5.2, 4.1, 3.8, 6.0, 8.0, 7.0, 6.5],
    [5.8, 4.3, 3.9, 6.2, 8.5, 7.2, 6.7],
    [8.1, 7.5, 6.8, 9.0, 11.0, 10.0, 9.5],
    [12.3, 11.5, 10.8, 13.5, 16.0, 15.0, 14.0]
  ],
  "marginal_time_labels": ["14:00", "16:00", "18:00", "20:00", "22:00", "00:00", "02:00"]
}

Rows are the 4 MARGINAL_EXTRA_KWH_LEVELS (1/2/4/8 kWh). Columns are the 7 MARGINAL_TIME_OFFSETS. Values are p/kWh, same as the existing HA sensor attribute.

Fallback behaviour

No marginal data (e.g. older PredBat instance) → empty lists are published.
Gateway firmware handles empty by falling back to its slot-aware RAG.

Extends the predbat_data MQTT payload with the marginal cost matrix produced by the Marginal mixin. The gateway's range display uses this to colour appliance RAG indicators based on the real cost of running each appliance now vs in upcoming time windows — replacing the slot-category heuristic that couldn't distinguish a 100W trickle from a full 3kW solar covering the load. Payload additions: marginal_costs — 4×N matrix, rows are 1/2/4/8 kWh extra load, columns are the time windows from marginal.py marginal_time_labels — HH:MM labels for each column Missing/failed reads publish empty lists, so older PredBat versions without marginal.py are handled gracefully on the gateway side. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Review feedback — three issues in the first cut: 1. CRITICAL: attribute="all" is not supported by ha.py get_state(); it would look up an attribute literally named "all" which doesn't exist, so the read silently returned None and marginal_costs was always empty in the payload. Use attribute="matrix" directly — marginal.py publishes the full matrix as that named attribute. 2. Narrow the exception catch to (TypeError, ValueError, AttributeError, KeyError) instead of the blanket Exception, per the recent reviewer feedback pattern on #3725. 3. Align warning log with the file's established "Warn: GatewayMQTT:" convention (every other warning in the file uses that prefix). Also removes the defensive two-level unwrap and the nested _row helper (both were cargo-culted around the broken read; now unnecessary). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

mgazza · 2026-04-12T08:47:07Z

Code review

Found 1 critical issue, already addressed in b8bf2fe:

attribute="all" is not a supported parameter of HAInterface.get_state() — it only accepts attribute=<specific_name> or raw=True. Passing "all" looks up an attribute literally named "all", which doesn't exist, so get_state_wrapper() returned None and the or {} fallback collapsed the matrix read to empty. The MQTT payload would silently publish empty marginal_costs / marginal_time_labels lists on every cycle. Fixed by reading the named attribute directly: attribute="matrix".

https://github.com/springfall2008/batpred/blob/b8bf2fe9/apps/predbat/gateway.py#L858-L898

Same commit also:

Narrows the exception catch to (TypeError, ValueError, AttributeError, KeyError) — consistent with the narrowing pattern you flagged on Gateway fixes #3725.
Aligns the warning log with the file's "Warn: GatewayMQTT:" convention.
Removes the nested _row helper and defensive two-level unwrap (both were artefacts of the broken read).

Not actioned but noted for the reviewer:

No test coverage added. Recent gateway PRs (Gateway fixes #3725, Gateway timezone fixes #3736, Gateway fixes for soc chart #3739, Fix gateway to use inverter ID #3777) all included tests/test_gateway.py changes — happy to add coverage if you'd like before merging.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

6 new tests on TestPublishPredbatData: - test_marginal_costs_nominal_matrix: int-keyed 4×3 matrix flattens in canonical 1/2/4/8 order. - test_marginal_costs_string_keys_work: string-keyed matrix (JSON round-trip) produces identical output. - test_marginal_costs_missing_sensor_empty_lists: missing sensor publishes empty lists (fallback path the gateway firmware relies on for its own RAG fallback). - test_marginal_costs_missing_row_padded_with_zeros: one absent level pads with 0 rather than collapsing the whole matrix. - test_marginal_costs_non_numeric_value_caught: non-numeric cells (e.g. "N/A") are caught by the narrow except and fall back to empty, rather than raising. - test_marginal_costs_non_dict_matrix_ignored: malformed sensor (list instead of dict) ignored cleanly. All 126 gateway tests pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

mgazza · 2026-04-12T09:17:04Z

Added test coverage in 0f9860c — 6 new tests on TestPublishPredbatData covering the nominal case, string-keyed matrix (JSON round-trip), missing sensor, missing row, non-numeric values, and malformed matrix shape. All 126 gateway tests pass.

Copilot

Pull request overview

Extends the GatewayMQTT._publish_predbat_data MQTT payload to include PredBat’s marginal energy cost matrix so the ESP32 gateway can render appliance RAG indicators based on actual marginal cost rather than slot category heuristics.

Changes:

Publish marginal_costs (4×N matrix) and marginal_time_labels alongside existing predbat_data fields.
Add gateway-side flattening/normalization logic to handle int vs string keys and missing/invalid data.
Add unit tests covering nominal, string-keyed, missing-sensor, missing-row padding, and invalid-matrix fallback behaviors.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
apps/predbat/gateway.py	Adds marginal matrix extraction/flattening and includes it in the MQTT `predbat_data` payload.
apps/predbat/tests/test_gateway.py	Adds tests validating the new marginal payload fields and key fallback behaviors.

+                        # Skip rows that don't match the established column shape.
+                        if time_labels and any(tl not in row for tl in time_labels):
+                            # Missing columns — pad with 0 rather than dropping the row.
+                            tmp_costs.append([round(float(row.get(tl, 0) or 0), 2) for tl in time_labels])
+                        else:
+                            tmp_costs.append([round(float(row.get(tl, 0) or 0), 2) for tl in time_labels])


Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Extends the ESP32 Gateway predbat_data MQTT payload to include PredBat’s marginal energy cost matrix (4 load levels × upcoming time windows) so the gateway firmware can color appliance indicators based on actual marginal cost rather than slot categories.

Changes:

Add marginal cost matrix extraction/flattening to GatewayMQTT._publish_predbat_data() and publish it as marginal_costs + marginal_time_labels.
Add unit tests covering nominal, missing, malformed, and partially missing marginal matrix inputs.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.

File	Description
`apps/predbat/gateway.py`	Adds marginal matrix read/flatten logic and includes it in the `predbat_data` payload.
`apps/predbat/tests/test_gateway.py`	Adds tests validating the published marginal payload structure and fallbacks.

+                "predbat.rates": "10.0",
+                "predbat.cost_today": "0",
+                "predbat.ppkwh_today": "10.0",
+                "predbat.marginal_energy_costs#matrix": matrix,
+            }


+                "predbat.rates": "10.0",
+                "predbat.cost_today": "0",
+                "predbat.ppkwh_today": "10.0",
+                "predbat.marginal_energy_costs#matrix": matrix,
+            }


+                "predbat.rates": "10.0",
+                "predbat.cost_today": "0",
+                "predbat.ppkwh_today": "10.0",
+                "predbat.marginal_energy_costs#matrix": matrix,
+            }


+                "predbat.rates": "10.0",
+                "predbat.cost_today": "0",
+                "predbat.ppkwh_today": "10.0",
+                "predbat.marginal_energy_costs#matrix": matrix,
+            }


Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR extends the MQTT predbat_data payload produced by GatewayMQTT._publish_predbat_data() to include PredBat’s marginal energy cost matrix (and corresponding time labels) so the ESP32 gateway can render appliance RAG indicators based on real marginal p/kWh impact rather than inferred slot categories.

Changes:

Add marginal_costs (4×N list) and marginal_time_labels (N labels) to the gateway MQTT payload, sourced from sensor.{prefix}_marginal_energy_costs attribute matrix.
Add gateway publish-path tests covering nominal matrix flattening, string/int keys, missing rows padding, and malformed data fallback behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
`apps/predbat/gateway.py`	Reads the marginal cost sensor matrix and publishes it as JSON lists in the gateway MQTT payload with defensive fallbacks.
`apps/predbat/tests/test_gateway.py`	Adds unit tests validating payload formatting and error handling for the marginal cost matrix fields.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

mgazza and others added 2 commits April 12, 2026 08:54

springfall2008 requested a review from Copilot April 14, 2026 19:46

Copilot started reviewing on behalf of springfall2008 April 14, 2026 19:46 View session

Copilot AI reviewed Apr 14, 2026

View reviewed changes

springfall2008 and others added 2 commits April 14, 2026 21:51

Potential fix for pull request finding

0831b30

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

879e566

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

springfall2008 requested a review from Copilot April 14, 2026 19:54

Copilot started reviewing on behalf of springfall2008 April 14, 2026 19:55 View session

Copilot AI reviewed Apr 14, 2026

View reviewed changes

springfall2008 and others added 3 commits April 14, 2026 22:04

Potential fix for pull request finding

a5c05ff

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

e0831ca

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

f2b5c69

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

springfall2008 requested a review from Copilot April 14, 2026 20:07

Copilot started reviewing on behalf of springfall2008 April 14, 2026 20:07 View session

Copilot AI reviewed Apr 14, 2026

View reviewed changes

Comment thread apps/predbat/tests/test_gateway.py Outdated

Potential fix for pull request finding

29fd555

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

springfall2008 merged commit 03bdd01 into main Apr 16, 2026
1 check passed

springfall2008 deleted the feat/gateway-marginal-costs branch April 16, 2026 18:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: publish marginal cost matrix to predbat gateway#3783

feat: publish marginal cost matrix to predbat gateway#3783
springfall2008 merged 9 commits intomainfrom
feat/gateway-marginal-costs

mgazza commented Apr 12, 2026

Uh oh!

mgazza commented Apr 12, 2026

Uh oh!

mgazza commented Apr 12, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

mgazza commented Apr 12, 2026

Summary

Payload additions

Fallback behaviour

Related

Uh oh!

mgazza commented Apr 12, 2026

Code review

Uh oh!

mgazza commented Apr 12, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants