Spatial complexity reduction for the run pipeline (partial) - #706
Merged
MoritzSchloesser merged 8 commits intoJul 15, 2026
Conversation
Adds spatial_reduce/spatial_restore tasks bracketing optimize, so the OPF
can run on a spatially-reduced grid while reinforcement still runs on the
full topology. New core function apply_reduced_results_to_full_grid maps
optimized flexible-component dispatch (charging points, heat pumps, DSM
loads, storage) back onto the full grid, by name or disaggregated onto
old_name members per time step (weighted by each member's pre-OPF
flexibility envelope) when aggregation_mode=True. Reactive power is
recomputed on restore, mirroring how pm_optimize itself handles it.
optimize now also records its flexible-component name lists on ctx.flags
and declares provides={"optimized_dispatch"}, so the validator's existing
requires/provides system enforces spatial_restore running after both
spatial_reduce and optimize.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
get_flexibility_bands only resamples bands to the active time-series FREQUENCY, not the active date range - so after a manual/short select_timesteps window, flexibility_bands kept spanning the full underlying SimBEV data range with no later step ever trimming it down. Any consumer indexing flexibility_bands by edisgo.timeseries.timeindex (e.g. the OPF's charging-point constraint builder) could then hit missing time steps. build_flexibility_bands now trims electromobility data to the active time index right after building the bands. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Standalone preset wiring spatial_reduce/spatial_restore around optimize, disabled by default via spatial_reduction.enabled. run_example_06.py runs it against a real ding0 grid. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CONTEXT.md gains a glossary section for spatial reduction terms (busmap, disaggregation rule, config surface, ordering). docs_notes/ carries the full design-session record plus two follow-up issue write-ups: the flexibility-bands time-index gap (fixed) and the aggregation_mode=True + charging-points gap (not yet fixed, tracked for later). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The preset's header comment and self-contained-run example still said "UC5"/uc5_spatial_reduction from when the file was copied from uc5_select_timesteps.yaml. Also adds a full issue description covering what's done and what's still open (aggregation_mode=True + charging points) for the spatial-reduction pipeline integration. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Hooks weren't run before the original commits landed. Adds the missing license header to edisgo/run/context.py, applies ruff-format's line collapsing across a few files, and manually wraps two docstring :func: cross-references that were still over the line-length limit after formatting. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
MoritzSchloesser
added a commit
that referenced
this pull request
Jul 16, 2026
* Change database from OEP to local
* feat: spatial complexity reduction for the run pipeline
Adds spatial_reduce/spatial_restore tasks bracketing optimize, so the OPF
can run on a spatially-reduced grid while reinforcement still runs on the
full topology. New core function apply_reduced_results_to_full_grid maps
optimized flexible-component dispatch (charging points, heat pumps, DSM
loads, storage) back onto the full grid, by name or disaggregated onto
old_name members per time step (weighted by each member's pre-OPF
flexibility envelope) when aggregation_mode=True. Reactive power is
recomputed on restore, mirroring how pm_optimize itself handles it.
optimize now also records its flexible-component name lists on ctx.flags
and declares provides={"optimized_dispatch"}, so the validator's existing
requires/provides system enforces spatial_restore running after both
spatial_reduce and optimize.
* fix: trim electromobility flexibility bands to the active time index
get_flexibility_bands only resamples bands to the active time-series
FREQUENCY, not the active date range - so after a manual/short
select_timesteps window, flexibility_bands kept spanning the full
underlying SimBEV data range with no later step ever trimming it down.
Any consumer indexing flexibility_bands by edisgo.timeseries.timeindex
(e.g. the OPF's charging-point constraint builder) could then hit missing
time steps. build_flexibility_bands now trims electromobility data to the
active time index right after building the bands.
* Add uc6_spatial_reduction preset and example runner
Standalone preset wiring spatial_reduce/spatial_restore around optimize,
disabled by default via spatial_reduction.enabled. run_example_06.py runs
it against a real ding0 grid.
* docs: spatial reduction glossary and grilling-session notes
CONTEXT.md gains a glossary section for spatial reduction terms (busmap,
disaggregation rule, config surface, ordering). docs_notes/ carries the
full design-session record plus two follow-up issue write-ups: the
flexibility-bands time-index gap (fixed) and the aggregation_mode=True +
charging-points gap (not yet fixed, tracked for later).
* Change overlying_grid path
* Fix stale UC5 naming in uc6_spatial_reduction.yaml, add issue writeup
The preset's header comment and self-contained-run example still said
"UC5"/uc5_spatial_reduction from when the file was copied from
uc5_select_timesteps.yaml. Also adds a full issue description covering
what's done and what's still open (aggregation_mode=True + charging
points) for the spatial-reduction pipeline integration.
* style: apply pre-commit hooks to the spatial-reduction commits
Hooks weren't run before the original commits landed. Adds the missing
license header to edisgo/run/context.py, applies ruff-format's line
collapsing across a few files, and manually wraps two docstring
:func: cross-references that were still over the line-length limit
after formatting.
---------
8 tasks
joda9
added a commit
that referenced
this pull request
Jul 29, 2026
…0.3.0 The changelog stopped at the documentation restructuring and covered none of the work merged since: the whole edisgo.run pipeline framework (#731), the spatial complexity reduction integration (#706), the OPF rework towards feasible non-optimal solutions and per-interval solving, the reworked database access, the get_flexibility_bands time-index scoping and the SPDX license headers (#607). Also sets the release date.
joda9
added a commit
that referenced
this pull request
Aug 3, 2026
* docs(whatsnew): document the run pipeline, OPF and database work for 0.3.0 The changelog stopped at the documentation restructuring and covered none of the work merged since: the whole edisgo.run pipeline framework (#731), the spatial complexity reduction integration (#706), the OPF rework towards feasible non-optimal solutions and per-interval solving, the reworked database access, the get_flexibility_bands time-index scoping and the SPDX license headers (#607). Also sets the release date. * chore(release): 0.3.0 metadata, authors and citation info - setup.py: version 0.3.0 (was 0.3.0dev, which normalised to 0.3.0.dev0), and fill in the metadata PyPI has been missing entirely: python_requires (>=3.10,<3.13 -- numpy is pinned to 1.26.4, which has no 3.13 wheels, and the test matrix covers 3.10 to 3.12), trove classifiers and project_urls. - authors: Jonas Danke and Moritz Schlösser are maintaining eDisGo now, so they lead the author field and are named as maintainers; the previous contributors stay listed. Maike Held was missing from it entirely despite being the second-largest contributor. - AUTHORS.md and CITATION.cff are new. Neither existed, so there was no complete credit list anywhere outside the git history, and GitHub had nothing to render for "Cite this repository". - .mailmap consolidates the identities people committed under: git counted 43 distinct authors for 19 actual people. Two commits were authored with Jonas' name but Kilian's address, so that entry matches on both fields rather than reassigning all of Kilian's commits. - doc/conf.py: extend the stale copyright year. Verified: twine check passes on both artifacts, the wheel METADATA carries the new author/maintainer/classifiers/Requires-Python, and CITATION.cff parses with all required CFF 1.2.0 fields present. * Update eDisGo_OPF.jl Manifest for Julia 1.12 compatibility StaticArrays 1.5.12 (pinned in the old manifest) fails to precompile on Julia >=1.11 due to Base's AbstractTriangular gaining a type parameter. Pkg.update() resolved newer, compatible versions of StaticArrays and its dependents (JuMP, PowerModels, MathOptInterface, Gurobi.jl, etc.). * Bump Julia to 1.12 in CI, compat and install docs The updated eDisGo_OPF.jl Manifest was resolved with Julia 1.12 and pins stdlibs that do not exist in earlier versions (e.g. JuliaSyntaxHighlighting, StyledStrings, LinearAlgebra 1.12). Main.jl runs Pkg.instantiate() on that checked-in manifest, so the CI job pinned to Julia 1.6 could not resolve it and test_pm_optimize failed. Keeping both 1.6 and 1.12 working is not possible with a single checked-in manifest, so the declared Julia version is raised everywhere it appears.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Partial solution to #705 (spatial complexity reduction pipeline
integration — see
docs_notes/issue_spatial_complexity_reduction_pipeline_integration.mdfor the full issue writeup).
Summary
Adds spatial complexity reduction to the eDisGo run pipeline, as a
counterpart to the existing temporal complexity reduction
(
select_timesteps). Spatial reduction merges nearby buses intorepresentative buses to shrink the grid the OPF runs on; reinforcement
still runs on the full topology.
spatial_reduce/spatial_restoretasks bracketingoptimize.spatial_reducestashes a deepcopy of the full grid and spatiallyreduces the working object;
spatial_restorewrites the optimizedflexible-component dispatch (charging points, heat pumps, DSM loads,
storage) back onto the stashed full grid and recomputes reactive power.
Both are no-ops when
spatial_reduction.enabledis false.apply_reduced_results_to_full_grid+EDisGo.map_reduced_results_to_full_grid, mirroring the existingspatial_complexity_reduction/EDisGo.spatial_complexity_reductionpairing.
requires/providescapabilitysystem (no new validator machinery):
spatial_reduceprovidesreduced_grid,optimizeadditionally providesoptimized_dispatch,spatial_restorerequires both.spatial_reduction:config block, mirroringtimeseries_selection:exactly — same shape, same eGo injection pattern(global default + per-grid override).
uc6_spatial_reduction.yamland eGo examplescenario_setting_uc6_example.json._build_run_edisgo_configinjectsspatial_reduction/spatial_reduction_per_gridthe same way it already injectstimeseries_selection.This PR covers
aggregation_mode=Falseonly.aggregation_mode=Truehas a known, documented gap unrelated to this PR's own logic — see
"Known limitation" below and the linked issue.
Testing
tests/tools/test_spatial_complexity_reduction.py::TestApplyReducedResultsToFullGrid— by-name write-back, disaggregation-sum exactness, the numpy-array-input
regression, the time-index-mismatch guard, and reactive-power recompute,
all against stub OPF results (no Julia/Gurobi dependency).
eGo/tests/tools/test_edisgo_integration.py— thespatial_reduction/spatial_reduction_per_gridinjection logic.eDisGo (
run_example_06.py) and through eGo(
scenario_setting_uc6_example.json):spatial_reduce → optimize (Gurobi) → spatial_restore → reinforce → saveall completed successfully.analyse_spatial_reduction.ipynb(not part of this PR, kept local) drivesthe pipeline task-by-task and confirms the reduced grid's OPF output
exactly matches the full grid's post-restore value for every flexible
component (0/388 mismatches on the test run).
Known limitation (tracked, not fixed in this PR)
aggregation_mode=Truemerges loads at the same bus and correctlyaggregates
loads_active_power, but does not aggregateelectromobility.flexibility_bands— the OPF's charging-point constraintbuilder reads that separately, keyed by charging-point name, so a merged
representative has no
upper_powerentry andoptimizeraisesKeyErrorbefore
spatial_restoreever runs. Only manifests withaggregation_mode: trueand 2+ charging points sharing a bus. Full writeup with the exacttraceback and suggested fix in
docs_notes/issue_aggregation_mode_flexibility_bands_not_aggregated.md.aggregation_mode: falseis the default in the new preset and unaffected.Also included
build_flexibility_bandsnow trimselectromobility.flexibility_bandsto the active time index right after building it — it previously only
resampled to the active frequency, not the active date range, so a
manual/short
select_timestepswindow left bands spanning the fullunderlying data range with nothing ever trimming them down. Unrelated to
spatial reduction directly, but was blocking end-to-end testing of it
(see
docs_notes/issue_temporal_reduction_flexibility_bands.md).flexible_* or []inapply_reduced_results_to_full_gridraisedValueErroron numpy-array input (task_optimizederivesflexible_loadsasedisgo.dsm.p_min.columns.values, not a list) —found on the first live end-to-end run.
Not in this PR
aggregation_mode=Truefix (tracked issue, linked above).spatial_reduce/spatial_restore(current scope is core-function-only, by design — seedocs_notes/spatial_reduction_grilling_session.md).optional"; "disaggregation by pre-OPF flexibility envelope") — not yet
written up.