Fix the PV calibration cap so it bounds the planner's forecast

[springfall2008]

Summary

The cap in pv_calibration() was meant to stop calibration scaling the solar forecast above what the array can physically produce. It wasn't doing that. Three defects:

1. It never reached the planner. The cap was applied to pv_estimateCL/pv_estimate10/pv_estimate90, which only annotate published sensor attributes. pv_calibration() returned the uncapped pv_forecast_minute_adjusted — the series the optimiser actually plans against. Since slot_adjustment is clamped to [0.2, 4.0] and average_day_scaling to [0.1, 2.0], and they compound, the planner could receive a forecast several times the array's ceiling.
2. It could clip below the raw forecast. min(max_kwh_cap, observed_cap) ignored what the forecast itself predicted for a slot, so a dull week's low observed peak suppressed the next sunny day's plan.
3. It was circular. observed_cap included max_pv_power_forecast, which is read back from the pv_forecast_h0 sensor — whose state is that same capped output.

The new cap

Per slot, in kWh per plan interval:

observed_slot = max_pv_power_hist / 60 * plan_interval_minutes
ceiling_slot  = max(1.2 * max_kwh, max_pv_power_hist) / 60 * plan_interval_minutes
raw_slot[s]   = sum of pv_forecast_minute over the slot     # pre-scaling forecast
cap[s]        = min(ceiling_slot, max(observed_slot, raw_slot[s]))

• 1.2 * max_kwh — declared array capacity plus headroom. max_kwh is kwp * efficiency summed across planes; it is not the inverter rating, contrary to the old comment. Inverter limits are the wrong bound for PV generation: a hybrid can harvest through the MPPT and DC-DC stage without crossing the AC rating, and for AC-coupled systems the battery inverter is a different device entirely. Cloud-edge enhancement and cool cells briefly push an array above nameplate, hence the 20%.
• max(..., max_pv_power_hist) in the ceiling — measured generation beats a declared figure. Under-declared kwp is common (users enter the inverter size, or one string of two), and the cap must never clip below what the meter recorded.
• max(observed_slot, raw_slot[s]) — the cap only limits scaling upwards; it never clips the raw forecast in the normal case.

Because ceiling_slot >= observed_slot and the inner max >= observed_slot, the invariant cap[s] >= observed_slot holds for all inputs.

max_pv_power_forecast is still computed and still logged — only its use in the cap is removed, which is what breaks the feedback loop.

Also fixed: the planner's P10 series

Capping P50 alone was not enough. pv_forecast_minute10 is rebuilt after the cap loop as P50 * worst_day_scaling, and worst_day_scaling had only a lower clamp while average_day_scaling is hard-clamped to 2.0. Whenever every day's actual/forecast ratio exceeds 2.0, the clamped denominator leaves worst_day_scaling > 1.0 — measured at up to 4x the cap, and inverted above P50, which silently disables cloud modelling (fetch.py:50 returns None when pv_total10 >= pv_total).

worst_day_scaling is now clamped to at most 1.0. A "worst day relative to average" multiplier above 1.0 is incoherent regardless, and since P10 derives from the already-capped P50 this makes P10 <= P50 <= cap hold by construction.

New diagnostic

The existing capped_slots log couldn't distinguish calibration scaling too hard (intended) from a config problem. A separate warning now fires when the raw forecast alone exceeds the ceiling:

Warn: SolarAPI: PV Calibration: Raw forecast exceeds the array ceiling in 12 slots
(peak 5.0kW vs 3.6kW ceiling) - check kwp and pv_scaling (currently 1.5),
forecast is being clipped to the ceiling

pv_scaling is deliberately not applied to the ceiling. It is a user dial applied upstream at solcast.py:1366; letting it move a physical bound would defeat the cap in the direction that matters — pv_scaling: 3.0 would yield a 3.6x nameplate ceiling. The warning points at kwp, which is the correct knob.

Behaviour changes

• Solcast and HA-sensor users have max_kwh = 9999, so the ceiling is inert and the cap reduces to max(observed_slot, raw_slot[s]). Both terms are floors, so no plan can be cut below the greater of the demonstrated peak and today's own raw forecast. What changes is that a plan which could previously reach 8x raw is now bounded. The new warning cannot fire for them.
• No usable history (new install, or recovery from an outage) leaves max_pv_power_hist = 0, so the ceiling is strictly 1.2 * max_kwh and a raw forecast above that is clipped. Intended, and pinned by a test.

Testing

Nine new tests in apps/predbat/tests/test_solcast.py covering: the raw-forecast floor, the headroom ceiling binding, an under-declared kwp not clipping observed generation, max_kwh = 9999, the cap reaching the planner's data, P10 never exceeding the cap or P50, the no-history ceiling behaviour, and the new warning firing and not firing. One existing test was updated — its fixture was precisely the case defect 2 describes.

Full suite passes: All tests passed (total time: 277.29s).

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR fixes PV forecast capping in SolarAPI.pv_calibration() so the planner is bounded by a per-slot physical ceiling (instead of only capping published sensor attributes), removes the circular dependency on the pv_forecast_h0 sensor-derived peak, and ensures the pessimistic P10 series cannot exceed P50/cap.

Changes:

• Reworked PV calibration capping to be computed per plan slot from min(ceiling_slot, max(observed_slot, raw_slot)) and applied to pv_forecast_minute_adjusted (planner-consumed data) via proportional scaling.
• Clamped worst_day_scaling to <= 1.0 to prevent P10 from exceeding P50/cap.
• Added/updated Solcast calibration tests to pin the new cap invariants and new warning behavior; added an implementation-plan doc.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
docs/superpowers/plans/2026-07-30-pv-calibration-cap.md	Adds an implementation plan and test/runbook instructions for the PV calibration cap changes.
apps/predbat/tests/test_solcast.py	Expands unit coverage for the new per-slot cap behavior, planner-data capping, P10 constraints, and warning emission.
apps/predbat/solcast.py	Applies a per-slot physical ceiling cap to planner forecast data and bounds worst-day scaling to keep P10 coherent.

Comments suppressed due to low confidence (8)

docs/superpowers/plans/2026-07-30-pv-calibration-cap.md:325

• This command uses a user-specific absolute path, which makes the instructions non-reproducible for other contributors. Prefer a repo-relative path (or a generic placeholder) in docs.

cd /Users/treforsouthwell/source/batpred/coverage

docs/superpowers/plans/2026-07-30-pv-calibration-cap.md:377

• This command uses a user-specific absolute path, which makes the instructions non-reproducible for other contributors. Prefer a repo-relative path (or a generic placeholder) in docs.

cd /Users/treforsouthwell/source/batpred/coverage

docs/superpowers/plans/2026-07-30-pv-calibration-cap.md:388

• This command uses a user-specific absolute path, which makes the instructions non-reproducible for other contributors. Prefer a repo-relative path (or a generic placeholder) in docs.

cd /Users/treforsouthwell/source/batpred

docs/superpowers/plans/2026-07-30-pv-calibration-cap.md:453

• This command uses a user-specific absolute path, which makes the instructions non-reproducible for other contributors. Prefer a repo-relative path (or a generic placeholder) in docs.

cd /Users/treforsouthwell/source/batpred/coverage

docs/superpowers/plans/2026-07-30-pv-calibration-cap.md:496

• This command uses a user-specific absolute path, which makes the instructions non-reproducible for other contributors. Prefer a repo-relative path (or a generic placeholder) in docs.

cd /Users/treforsouthwell/source/batpred/coverage

docs/superpowers/plans/2026-07-30-pv-calibration-cap.md:509

• This command uses a user-specific absolute path, which makes the instructions non-reproducible for other contributors. Prefer a repo-relative path (or a generic placeholder) in docs.

cd /Users/treforsouthwell/source/batpred/coverage

docs/superpowers/plans/2026-07-30-pv-calibration-cap.md:521

• This command uses a user-specific absolute path, which makes the instructions non-reproducible for other contributors. Prefer a repo-relative path (or a generic placeholder) in docs.

cd /Users/treforsouthwell/source/batpred

docs/superpowers/plans/2026-07-30-pv-calibration-cap.md:533

• This command uses a user-specific absolute path, which makes the instructions non-reproducible for other contributors. Prefer a repo-relative path (or a generic placeholder) in docs.

cd /Users/treforsouthwell/source/batpred/coverage

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

apps/predbat/solcast.py:1137

• pv_estimate10/pv_estimate90 are computed from the pre-scale-down pv_value, then pv_forecast_minute_adjusted is potentially scaled down to capped_data. In slots where pv_value > capped_data and worst_day_scaling < 1.0, this makes the published pv_estimate10 inconsistent with the planner’s P10 series (built later from the scaled-down P50) and can overstate the pessimistic forecast (e.g. pv_value=10, cap=3, worst=0.7 -> pv_estimate10 becomes 3.0 but the planner’s P10 would be 2.1). Derive pv_estimate10/90 from the capped P50 slot value instead.

            capped_data = min(ceiling_slot, max(observed_slot, raw_value))
            pv_estimateCL[minute] = dp4(min(pv_value, capped_data))
            pv_estimate10[minute] = dp4(min(pv_value * worst_day_scaling, capped_data))
            pv_estimate90[minute] = dp4(min(pv_value * best_day_scaling, capped_data))