chore: pin dependencies exactly and settle the resolution decision - #7
Merged
Conversation
Two Phase 0 loose ends, both about making later phases verifiable. Dependencies are now pinned with == across all extras, matching solaredge2mqtt and learninghouse. Updates arrive as individual dependabot pull requests that run the full suite, rather than silently on whatever day a transitive resolve changes. scikit-learn is the only pin that is load-bearing rather than tidy. Verified empirically: the frozen baseline reproduces bit-identically across numpy 2.4.6/2.5.1, pandas 3.0.3/3.0.5 and scipy 1.17.1/1.18.0 as long as scikit-learn stays at 1.9.0. A scikit-learn bump therefore has to regenerate the baseline and say so in the changelog; the others do not. The forecast resolution question from chapter 6 is decided and written up as a new chapter 3.5. The MVP computes hourly; the data model keeps finer resolutions open without implementing them. The interval is a brain property, constrained by both the provider and what the client can push, not a provider property as originally framed. What becomes interval-aware now, because retrofitting it would invalidate every trained model: interval as a required field in brain config and model metadata with invalidation on mismatch, time features on minutes-since-midnight rather than hour, power_period derived from the interval instead of a fixed hour, and aggregation that sums intervals within a period. Why not implement it now: there are no sub-hourly measurements to train on and no baseline for such a path, OpenWeatherMap users are permanently hourly so it would mean two code paths before the first release, and Open-Meteo serves minutely_15 natively only in Central Europe and North America — elsewhere it interpolates hourly data, which yields four times the rows with the same information and strongly autocorrelated neighbours that flatter any naive holdout. Chapter 3.3 also records the Phase 0 measurement that justifies dropping the power model: MAE 620.88 Wh against 624.93 W, R2 0.886 against 0.885. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Johannes Ott <mail@johannes-ott.net>
There was a problem hiding this comment.
Pull request overview
Pins runtime/dev/service dependencies to exact versions in pyproject.toml and documents two Phase 0 architecture decisions in pvlearn-umsetzungsplan.md (dependency pinning rationale + forecast interval/resolution as a brain property), aiming to make later phases verifiable and avoid silent model drift.
Changes:
- Replace loose dependency ranges with exact
==pins for core +dev+serviceextras, and document why scikit-learn is treated as reproducibility-critical. - Extend the roadmap/data-model notes with
interval_minutesin model metadata and a decided “MVP hourly, model open for finer” resolution policy. - Record the empirical justification for dropping the separate power model and describe interval-aware aggregation/power derivation.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| pyproject.toml | Switches dependencies and extras to exact pins and adds rationale comments for reproducibility. |
| pvlearn-umsetzungsplan.md | Updates roadmap/spec text around interval-aware modeling, metadata invalidation, and model target semantics. |
Suppressed comments (2)
pvlearn-umsetzungsplan.md:121
- This paragraph says models are invalidated on the "sklearn minor version" but immediately below states reproducibility depends on the exact scikit-learn version. The invalidation rule should be described consistently (compare exact version, or adjust the explanation).
Beim Laden gilt hart: **Stimmt `feature_schema_version`, die sklearn-Minor-Version, der Provider, das Intervall oder die Location nicht überein, wird das Modell verworfen und neu trainiert.** Kein Migrationsversuch, kein Best-Effort-Laden. Stumme Fehlprognosen durch ein Modell, das auf einem anderen Feature-Set trainiert wurde, sind praktisch nicht debugbar.
Zur sklearn-Version: die Phase-0-Baseline ist nur gegen exakt die Version reproduzierbar, unter der sie entstanden ist. `pvlearn` pinnt scikit-learn deshalb exakt (siehe 6.6); ein Bump verschiebt still jede Prognose und erfordert eine neu erzeugte Baseline.
pvlearn-umsetzungsplan.md:140
- Section 3.5 refers to
intervalas a required config/metadata field, but the metadata schema example usesinterval_minutes. Using one name consistently (or explicitly defining both) will avoid divergent implementations later.
- `interval` als Pflichtfeld in Brain-Konfiguration und Modell-Metadaten, mit Invalidierung bei Abweichung (siehe 3.4)
Comment on lines
+22
to
+26
| # Every dependency is pinned exactly, matching solaredge2mqtt and learninghouse. | ||
| # Updates arrive as individual dependabot pull requests that run the full test | ||
| # suite, rather than silently on whatever day a transitive resolve changes. | ||
| # | ||
| # scikit-learn is the one pin that is load-bearing rather than merely tidy: the |
| `power_period` wird **weiterhin publiziert**, abgeleitet als `energy_wh / 1h`. Damit ist der Wegfall kein Breaking Change für MQTT-Konsumenten. Was verloren geht, ist die Momentanleistung zum Zeitstempel; das braucht weder das Energy Dashboard noch ein bekannter Automations-Use-Case. Im Changelog als „jetzt Stundenmittel statt Momentanwert" dokumentieren. | ||
| Auf dem Referenzdatensatz aus Phase 0 ist das empirisch bestätigt: MAE 620,88 Wh für das Energiemodell gegenüber 624,93 W für das Leistungsmodell, R² 0,886 gegenüber 0,885. Die beiden Modelle liegen unter einem Prozent auseinander. | ||
|
|
||
| `power_period` wird **weiterhin publiziert**, abgeleitet als `energy_wh / interval`. Damit ist der Wegfall kein Breaking Change für MQTT-Konsumenten. Was verloren geht, ist die Momentanleistung zum Zeitstempel; das braucht weder das Energy Dashboard noch ein bekannter Automations-Use-Case. Im Changelog als „jetzt Intervallmittel statt Momentanwert" dokumentieren. |
Pinning numpy and scipy exactly forced the choice: numpy 2.5 and scipy 1.18 both require Python 3.12, so supporting 3.11 would mean holding both a minor version back indefinitely. Supported versions are now 3.12 and 3.13. build-check covers 3.13, the compat matrix covers 3.12, so both remain tested on every run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Johannes Ott <mail@johannes-ott.net>
- power_period was written as energy_wh / interval while the metadata field is interval_minutes, which is exactly the ambiguity the surrounding paragraph warns about. It is energy_wh * 60 / interval_minutes. - The model metadata example still showed sklearn_version 1.5.2 while this branch pins 1.9.0 and calls that pin load-bearing. - The pinning comment claimed every dependency is pinned, but the build-system requirements stay on lower bounds. Narrowed the claim and recorded why they are treated differently. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Johannes Ott <mail@johannes-ott.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two Phase 0 loose ends, both about making later phases verifiable.
Dependency pinning
All dependencies are now pinned with
==across every extra, matchingsolaredge2mqttandlearninghouse. Updates arrive as individual dependabot pull requests that run the full suite, rather than silently on whatever day a transitive resolve changes.scikit-learn is the only pin that is load-bearing rather than merely tidy, and this was verified rather than assumed. The frozen baseline was regenerated under the newer libraries and compared column by column:
So a scikit-learn bump has to regenerate the baseline and say so in its changelog; the others do not. That asymmetry is now written down where the pins are.
This closes open point 6 of the roadmap, which this branch had opened one commit earlier.
Forecast resolution — chapter 3.5
Open point 1 is decided and written up. The MVP computes hourly; the data model keeps finer resolutions open without implementing them.
The original framing — resolution per weather provider — does not hold. The interval is a brain property, constrained from both sides:
Open-Meteo can do 15 minutes, but not if the inverter only reports hourly. OpenWeatherMap cannot do it at all. Modelled as a provider attribute, contradictory states appear immediately.
Interval-aware now, because retrofitting any of it invalidates every trained model:
intervalas a required field in brain config and model metadata, with invalidation on mismatchhour_sin/coswould otherwise be equal for all intervals within an hourpower_periodderived from the interval instead of a fixed hour, which is what keeps a later switch from becoming a silent factor-4 errorNot implemented now, for reasons independent of the architecture: there are no sub-hourly measurements to train on and therefore no baseline for such a path; OpenWeatherMap users are permanently hourly, so it would mean two code paths before the first release; and Open-Meteo serves
minutely_15natively only in Central Europe (ICON-D2, AROME) and North America (HRRR) — elsewhere it interpolates hourly data, giving four times the rows with the same information and strongly autocorrelated neighbours that flatter any naive holdout. Chapter 7 already lists chapter-6 scope creep as the risk that stops the project reaching a release.Also
Chapter 3.3 now records the Phase 0 measurement that justifies dropping the power model: MAE 620.88 Wh against 624.93 W, R² 0.886 against 0.885 — under one percent apart, exactly as the chapter argued from first principles.
Decided points stay in chapter 6 struck through with a pointer to the reasoning, rather than being deleted.
Test plan
pyproject.tomlresolves cleanlyruff check ./ruff format --check .— cleanpyright— 0 errorspytest --cov=pvlearn— 26 passed, coverage gate satisfiedThe 3.11 and 3.12 compat jobs are the real check on whether the exact pins hold across the supported Python range.
🤖 Generated with Claude Code