PyPSA reaches into linopy internals in several more places (follow-up to #741)

[FBumann]

Summary

Following #741 (PyPSA reaching into .vars/_term to spell "which rows have no terms", now fixed by the public has_terms), I audited PyPSA master (pinned at commit 30e4ed0) for other spots where it depends on linopy's internal representation rather than a public API.

"Reaching into internals" here means depending on the term-storage layout — .data (the raw backing Dataset), the vars/coeffs/const fields, the _term helper dim, the -1/FILL_VALUE dead-slot sentinel — or on low-level constructors and submodule import paths, rather than going through a stable public surface. Each of these couples PyPSA to how linopy happens to store things today and breaks silently when we refactor (cf. the recent linopy.alignment move in #742).

Below is the full list, ordered roughly by how deep the coupling is. Line numbers are permalinks into PyPSA at the pinned commit.

---

0. (lhs.vars == -1).all("_term") — already fixed by #741

constraints.py#L1166. Listed for completeness — the new has_terms property replaces it. The remaining items are the same class of problem in other places.

---

1. Importing the FILL_VALUE dead-slot sentinel — linopy.variables.FILL_VALUE

constraints.py#L1735 and constraints.py#L1912:

previous_soc_pp = previous_soc_pp.where(
    include_previous_soc_pp.values, linopy.variables.FILL_VALUE
)

PyPSA reaches into the module-level FILL_VALUE = {"labels": -1, "lower": np.nan, "upper": np.nan} dict to hand-fill the dead slots of a Variable's raw Dataset after a manual roll/concat. This is the deepest coupling in the list — it hard-depends both on the -1 label sentinel and on the exact field names (labels/lower/upper) of the variable storage. This is the same -1-sentinel knowledge that #741 was about, but applied to variables and pulled straight from a private module constant.

2. Dropping to .data (the raw backing Dataset) for operations linopy doesn't forward

linopy doesn't expose roll, period-wise multi-index .sel, or reindex_like on Variable/LinearExpression, so PyPSA drops to the underlying Dataset and operates on it directly:

• constraints.py#L1711 — soc.data.sel(snapshot=(p, sl)).roll(snapshot=1) (a Variable)
• constraints.py#L1895 — e.data.sel(snapshot=(p, sl)).roll(snapshot=1) (a Variable)
• global_constraints.py#L267 — max_absolute_growth.reindex_like(lhs.data) (reindexing an external array onto a LinearExpression's coords)

The two *.data...roll(...) cases in §2 are exactly what then forces the manual FILL_VALUE rebuild in §1 — they're one workaround. The comments there (# Normally, we should use groupby, but is broken for multi-index, xarray issue #6836) show this is a reluctant workaround, not intended usage.

3. .const + .shape to test "is this expression empty / all-zero?"

expressions.py#L101:

def _aggregate_components_skip_iteration(self, vals: Any) -> bool:
    return vals is None or (not np.prod(vals.shape) and (vals.const == 0).all())

This depends on .shape (whose product folds in the _term helper dim) and on the const field to decide an expression is empty/trivial. The intended public predicate, expr.empty, is deprecated, and there's no clean "has no live terms and no nonzero constant" spelling — which is the same gap #741 identified, just from the other side.

4. Low-level LinearExpression(data, model) construction instead of a public factory

PyPSA builds expressions via the two-arg constructor, passing a raw constant (a DataFrame from get_switchable_as_dense, an ndarray) or None:

• expressions.py#L128 — LinearExpression(None, self._n.model) (empty expression)
• expressions.py#L164, #L219, #L291, #L690 — LinearExpression(<constant>, m)
• constraints.py#L820, #L853, #L869 — linopy.LinearExpression(<constant>, m)

The constant cases are precisely what the public LinearExpression.from_constant(model, constant) factory is for; the None (empty) case has no public spelling at all. Borderline — the constructor does support these inputs — but it leans on the low-level signature rather than the documented factory.

5. Public symbols imported through internal module paths (minor)

These import public names, but via submodule paths instead of the top-level package, so they break on internal reorg (e.g. the linopy.alignment move in #742):

• global_constraints.py#L15 — from linopy.expressions import merge (top-level from linopy import merge exists and is used elsewhere in PyPSA, e.g. constraints.py:16)
• optimize.py#L18 — from linopy.solvers import available_solvers (available_solvers is exported from top-level linopy)

---

Suggested public-API additions on the linopy side

To let PyPSA drop the internals, in rough priority order:

1. Variable/Expression masking without the sentinel — a public way to blank out slots (and to reason about live vs dead slots) so §1's FILL_VALUE import and §3's const/shape probing aren't needed. has_terms (Add public per-slot emptiness check on expressions (has_terms) — PyPSA reaches into .vars/_term internals #741) is the start; an equivalent for Variable, plus a public "fill/mask dead slots" helper, would close §1.
2. Forward roll / reindex_like (and clean multi-index .sel) onto Variable/LinearExpression so §2 doesn't need .data. These are the operations PyPSA explicitly works around today.
3. Encourage LinearExpression.from_constant (and add a public empty-expression constructor) to retire §4.
4. (Docs/lint) keep public symbols (merge, available_solvers) imported from top-level linopy to retire §5.

Happy to file the corresponding PyPSA-side cleanup PRs once the public equivalents land. Numbers were collected from PyPSA master at 30e4ed0.

[FBumann]

Closing as a duplicate. This overlaps with PyPSA/pypsa#1718 ("Remove remaining reliance on linopy internals (sentinels, private dims, FILL_VALUE)"), which already tracks the same findings — and does so more accurately: its public-API classification table correctly rules out the .const/.shape empty-check and the LinearExpression(...) construction that this issue flagged. Any linopy-side public-API work (a FILL_VALUE-free masking/fill helper, forwarding roll/reindex_like onto Variable/LinearExpression) can be tracked from #1718's checklist, which already anticipates a corresponding linopy issue.