fix(constraints): CSRConstraint stores label columns; frozen constraints survive later variable changes by FabianHofmann · Pull Request #927 · PyPSA/linopy

FabianHofmann · 2026-09-04T11:44:30Z

(handwritten intent goes here)

Note

The following content was generated by AI (Claude Code), reviewed by @FabianHofmann.

Fixes #926 (points 1, 3 and 5; points 2 and 4 depend on #870).

CSRConstraint stored dense variable positions as CSR column indices with a width fixed at freeze time. Any add_variables or remove_variables after a freeze=True constraint made model.matrices fail with inconsistent shapes.

Changes

CSRConstraint stores raw variable labels as columns (width model._xCounter). A shared helper maps labels to dense positions once at matrix assembly, for both the dense Constraint and the frozen path, and raises if a constraint references a removed variable.
The returned block keeps the stored data and indptr arrays by identity and preserves the int32 index dtype, so the persistent snapshot diff can still skip untouched frozen constraints and index arrays do not double in size.
netcdf: the writer tags _csr_columns: labels; the reader maps legacy position files back through the stored vlabels and raises on unknown tags.
Separate commit: CSRConstraint.indexes built an xarray Indexes without a variables mapping, which made ModelSnapshot.capture fail on any model with a frozen constraint.

Tests

Add/remove variables after freeze against the unfrozen model, netcdf round-trip incl. legacy layout, buffer identity and empty diff for an untouched frozen model.

Verification

Check	Result
`ruff check` / `ruff format`	clean
`mypy linopy`	28 errors, all pre-existing in `solvers.py`
`pytest -k "not xpress" --ignore=test/remote`	6350 passed, 532 skipped

…s at matrix assembly (#926)

codspeed-hq · 2026-09-04T11:49:30Z

Merging this PR will regress 3 benchmarks

⚠️

Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 12 improved benchmarks
❌ 3 regressed benchmarks
✅ 160 untouched benchmarks
🆕 6 new benchmarks
⏩ 175 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	`BASE`	`HEAD`	Efficiency
❌	`test_to_lp[sparse_network-n=10]`	718.8 KB	1,338.5 KB	-46.3%
❌	`test_to_lp[expression_arithmetic-n=10]`	718.4 KB	1,337.6 KB	-46.29%
❌	`test_to_lp[nodal_balance-severity=0]`	3.3 MB	3.7 MB	-11.33%
⚡	`test_to_solver[highs-rolling-severity=100]`	245 MB	186.1 MB	+31.65%
⚡	`test_to_solver[highs-rolling-severity=50]`	183.9 MB	140.2 MB	+31.1%
⚡	`test_to_solver[gurobi-rolling-severity=100]`	245.3 MB	188.3 MB	+30.24%
⚡	`test_to_solver[highs-kvl_cycles-severity=0]`	309.9 MB	242.8 MB	+27.64%
⚡	`test_to_solver[highs-cumsum-severity=100]`	79.5 MB	63.4 MB	+25.52%
⚡	`test_to_solver[highs-kvl_cycles-severity=50]`	158.1 MB	128.9 MB	+22.62%
⚡	`test_to_solver[gurobi-rolling-severity=50]`	184.1 MB	153.8 MB	+19.7%
⚡	`test_to_solver[highs-cumsum-severity=50]`	20.2 MB	17.1 MB	+18.58%
⚡	`test_to_solver[gurobi-kvl_cycles-severity=50]`	158.4 MB	134.2 MB	+18.04%
⚡	`test_to_solver[gurobi-kvl_cycles-severity=0]`	310.1 MB	268.9 MB	+15.32%
⚡	`test_to_solver[gurobi-cumsum-severity=50]`	20.5 MB	18 MB	+13.54%
⚡	`test_to_solver[gurobi-cumsum-severity=100]`	79.8 MB	70.6 MB	+12.96%
🆕	`test_build[nodal_balance_sparse-severity=0]`	N/A	1.5 MB	N/A
🆕	`test_build[nodal_balance_sparse-severity=100]`	N/A	1.5 MB	N/A
🆕	`test_build[nodal_balance_sparse-severity=50]`	N/A	1.5 MB	N/A
🆕	`test_to_lp[nodal_balance_sparse-severity=0]`	N/A	3.7 MB	N/A
🆕	`test_to_lp[nodal_balance_sparse-severity=100]`	N/A	3.7 MB	N/A
...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing fix/csr-label-columns (d338857) with master (6b133d5)²}

175 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
No successful run was found on master (b71e9a9) during the generation of this report, so 6b133d5 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

github-actions · 2026-09-04T11:49:45Z

Build cost — v1 vs legacy

v1 build peak & time relative to legacy, on this commit — not a comparison against master (that is CodSpeed).

peak — v1 / legacy	time — v1 / legacy

Full table (time + peak, mean)

benchmarks/drivers/test_build.py::test_build[basic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.06721 (1.13)   │   15.03 (1.00) 
 (v1)        0.05962 (1.0)   │    15.00 (1.0) 

benchmarks/drivers/test_build.py::test_build[basic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.06943 (1.06)   │   12.04 (1.00) 
 (v1)        0.06575 (1.0)   │    12.04 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=0]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.02807 (1.08)   │   15.20 (1.0) 
 (v1)        0.02592 (1.0)   │   15.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=100]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04411 (1.14)   │   44.93 (1.0) 
 (v1)         0.0388 (1.0)   │   44.93 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03107 (1.05)   │   11.51 (1.0) 
 (v1)        0.02964 (1.0)   │   11.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.07212 (1.04)   │   24.34 (1.06) 
 (v1)        0.06942 (1.0)   │    23.05 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=250]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.0823 (1.05)   │   16.12 (1.00) 
 (v1)       0.07825 (1.0)   │    16.12 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=10000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.01765 (1.00)   │   752.18 (1.10) 
 (v1)        0.01757 (1.0)   │    685.15 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=100]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.01736 (1.07)   │   3.12 (1.33) 
 (v1)        0.01621 (1.0)   │    2.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=0]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05057 (1.26)   │   126.16 (1.44) 
 (v1)        0.04029 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=100]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04933 (1.26)   │   126.16 (1.44) 
 (v1)        0.03927 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=50]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04961 (1.24)   │   126.16 (1.44) 
 (v1)        0.04004 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[masked-n=100]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.0405 (1.02)   │    715.14 (1.0) 
 (v1)       0.03977 (1.0)   │   787.74 (1.10) 

benchmarks/drivers/test_build.py::test_build[masked-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03875 (1.07)   │   4.55 (1.27) 
 (v1)        0.03628 (1.0)   │    3.59 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=0]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.2742 (1.07)   │   704.12 (1.09) 
 (v1)        0.2564 (1.0)   │    647.38 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=100]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.2927 (1.04)   │   18.34 (1.0) 
 (v1)        0.2821 (1.0)   │   18.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=50]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)   0.2859 (1.01)   │   9.54 (1.0) 
 (v1)        0.2826 (1.0)   │   9.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.05486 (1.07)   │   3.77 (1.12) 
 (v1)        0.05107 (1.0)   │    3.37 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=50]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05573 (1.09)   │   216.59 (1.10) 
 (v1)        0.05128 (1.0)   │    196.23 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=0]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.02848 (1.11)   │   938.49 (1.0) 
 (v1)        0.02577 (1.0)   │   938.49 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=100]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)   0.0312 (1.15)   │   9.66 (1.0) 
 (v1)       0.02715 (1.0)   │   9.66 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.02948 (1.09)   │   5.32 (1.0) 
 (v1)        0.02693 (1.0)   │   5.32 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=0]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.01626 (1.03)   │   1.41 (1.0) 
 (v1)        0.01586 (1.0)   │   1.41 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=100]
                time (s)       peak (MiB) 
 name               mean   │         mean 
──────────────────────────────────────────
 (legacy)   0.016 (1.01)   │   1.41 (1.0) 
 (v1)       0.0158 (1.0)   │   1.41 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance_sparse-severity=50]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)    0.016 (1.02)   │   1.41 (1.0) 
 (v1)       0.01565 (1.0)   │   1.41 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=1000]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.1377 (1.05)   │   946.85 (1.06) 
 (v1)        0.1315 (1.0)   │    891.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=10]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.1357 (1.05)   │   12.01 (1.0) 
 (v1)        0.1297 (1.0)   │   12.01 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03549 (1.08)   │   147.70 (1.06) 
 (v1)         0.0329 (1.0)   │    139.87 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03534 (1.09)   │   2.60 (1.09) 
 (v1)        0.03254 (1.0)   │    2.38 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=0]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03101 (1.19)   │   696.75 (1.03) 
 (v1)        0.02617 (1.0)   │    673.70 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=100]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.0707 (1.08)   │   137.97 (1.0) 
 (v1)       0.06552 (1.0)   │   137.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04805 (1.12)   │   69.22 (1.0) 
 (v1)        0.04272 (1.0)   │   69.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03393 (1.13)   │   402.33 (1.00) 
 (v1)         0.0301 (1.0)   │    402.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03288 (1.10)   │   3.19 (1.19) 
 (v1)           0.03 (1.0)   │    2.69 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.03699 (1.10)   │   29.00 (1.54) 
 (v1)        0.03351 (1.0)   │    18.84 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.04599 (1.12)   │   37.95 (1.43) 
 (v1)        0.04113 (1.0)   │    26.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[storage-n=10]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)    0.06941 (1.0)   │    410.93 (1.0) 
 (v1)       0.06995 (1.01)   │   427.84 (1.04) 

benchmarks/drivers/test_build.py::test_build[storage-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)    0.07591 (1.0)   │     9.94 (1.0) 
 (v1)       0.07727 (1.02)   │   10.22 (1.03)

📊 Interactive plots + CSV: download the semantics-report-v1-vs-legacy artifact from this run.

_{Report-only · not a gate · refreshed on every push · obsolete once legacy is dropped.}

FabianHofmann added 3 commits September 4, 2026 13:43

fix(constraints): CSRConstraint.indexes builds a valid xarray Indexes

14e1abe

fix(constraints): CSRConstraint stores label columns; map to position…

9d67125

…s at matrix assembly (#926)

Merge branch 'master' into fix/csr-label-columns

d338857

Navigation Menu

Uh oh!

fix(constraints): CSRConstraint stores label columns; frozen constraints survive later variable changes - #927

fix(constraints): CSRConstraint stores label columns; frozen constraints survive later variable changes#927
FabianHofmann wants to merge 3 commits into
masterfrom
fix/csr-label-columns

FabianHofmann commented Sep 4, 2026

Uh oh!

codspeed-hq Bot commented Sep 4, 2026

Uh oh!

github-actions Bot commented Sep 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

FabianHofmann commented Sep 4, 2026

Uh oh!

codspeed-hq Bot commented Sep 4, 2026

Merging this PR will regress 3 benchmarks

Performance Changes

Footnotes

Uh oh!

github-actions Bot commented Sep 4, 2026

Build cost — v1 vs legacy

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant