feat(compat): answer linopy v1 where the position knows the answer

[FBumann]

Based on main, independent of #189 — this half of the v1 work touches no
engine, so it lands whichever of duckdb or polars wins.

Makes the oracle lane run clean under linopy's v1 arithmetic convention
(PyPSA/linopy#717). No behaviour change on either lane, and no dependency
pin: it converts an inherited legacy accident into a stated intent.

The problem it fixes

load_parameters reindexes every parameter to the master coordinates, so a
parameter covering a subset of its dims reaches linopy as NaN. Under v1 that
raises:

ValueError: NaN found in a user-supplied constant. linopy treats this as ambiguous:
if you meant a *data error*, fix it with .fillna(value); if you meant *absent at
this slot*, mark it on the variable instead (mask=, .where(cond), .reindex(...))

It is right to refuse — from inside linopy a deliberate absence and a data error
are indistinguishable. But that is every sparse parameter: SPEC §8's "sparse
data gives sparse variables", and what a generated binder emits by the tableful.

The suite passing beforehand was not evidence against this — it had no sparse
float parameter. It has one now, and it fails without this change.

The shape of the fix

Per position, not once in the loader. The same missing row means three
different things, and only the evaluator knows which:

position	a missing row means	how
coefficient (w * x)	zero	semantics.coefficient
bounds:	nothing — it is an error	left NaN; raises as before
where operand	false	left NaN; SPEC §6's "non-null and finite"

A single fill in load_parameters has to pick one and is wrong for the other
two: an unbounded variable is not a variable bounded at zero, and a mask that
reads true everywhere is not a mask. The loader already made exactly this call
for booleans — fill_value=False, one line above the float branch.

linopy/semantics.py is the new home, mirroring linopy's own module of that
name and for its stated reason: the evaluator stays about evaluating, and a
later change to the convention is a single-file diff rather than a hunt.

semantics.vacated does the same for shift's edge. SPEC §7 declares that
vacated positions contribute zero; v1 §4 counts .shift() among the operations
that create absence, so without saying so an acyclic recurrence would silently
lose its first step. Saying it is v1 §7 working as intended — the caller
resolving absence at the call site.

Both are correct under the legacy convention too — they resolve absence to the
value legacy reached implicitly — so neither is conditional on the option. What
v1 changed is that staying silent stopped being available.

Verification

configuration	result
released linopy (as CI runs today)	407 passed, 1 xfailed
PyPSA/linopy#717, semantics="v1"	407 passed, 1 xfailed

No pin changed: to_linexpr().fillna(0) behaves identically across the released
line and the v1 branch, so nothing here requires depending on an unmerged PR.

Relationship to #234

#234 is the other half, on top of #189: adopting v1's absence semantics as the
language's own (a masked variable's term drops the row rather than contributing
zero), plus defined(v) and the v1 pin. That half is engine-coupled and
breaking. This one is neither, and is worth landing first either way.

Summary by CodeRabbit

• Bug Fixes

  • Corrected handling of missing or partially specified coefficients, variables, and bounds so uncovered entries behave as zero.
  • Ensured shifted and masked values preserve consistent absence semantics.

• Tests

  • Added coverage confirming sparse coefficients produce matching results across relational and eager execution paths.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds a linopy semantics module for treating missing coefficients, variables, and shifted positions as zero. The builder applies these semantics during AST and helper evaluation, with documentation and a differential regression test for sparse parameter dimensions.

Changes

Linopy semantics

Layer / File(s)	Summary
Define linopy absence semantics src/farkas/linopy/semantics.py, ARCHITECTURE.md	Adds coefficient, present, and vacated helpers and documents the module as the linopy v1 arithmetic-semantics location.
Apply semantics during evaluation src/farkas/linopy/builder.py, tests/test_relational.py	Wraps variables, parameters, and shifted arrays with the semantic helpers, and tests zero behavior for sparse coefficient and bound dimensions on both execution lanes.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is concise and clearly points to the Linopy v1 compatibility/positional semantics change in this PR.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/linopy-v1-compat

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/farkas/linopy/builder.py`:
- Around line 310-311: Update ParameterNode handling in the builder’s evaluator
to preserve missing parameter values by removing coefficient-style zero filling
from semantics.coefficient. Add fillna(0.0) only at expression-coefficient join
points where coefficient evaluation requires it. In tests/test_relational.py
lines 723-731, retain c with t=0 for the bound case and add or adjust separate
coverage for missing bound parameters to verify they raise.

In `@src/farkas/linopy/semantics.py`:
- Around line 39-44: Replace the Any annotations in the exported helpers
coefficient, present, and vacated with narrow protocols or supported public
types describing the required fillna() and to_linexpr() operations and their
concrete return values. Preserve each helper’s existing behavior while ensuring
Pyrefly can type-check all inputs and outputs without widening to Any.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ae80b510-b29a-4784-8df0-dd3cc3bc6ae3

📥 Commits

Reviewing files that changed from the base of the PR and between 9f3f73b and 5676e02.

📒 Files selected for processing (4)

• ARCHITECTURE.md
• src/farkas/linopy/builder.py
• src/farkas/linopy/semantics.py
• tests/test_relational.py

[coderabbitai]

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== files =="
git ls-files | rg '(^|/)builder\.py$|test_relational\.py$|src/farkas/linopy|tests/test_relational' || true

echo "== builder outline =="
if [ -f src/farkas/linopy/builder.py ]; then
  wc -l src/farkas/linopy/builder.py
  ast-grep outline src/farkas/linopy/builder.py --view expanded | sed -n '1,220p' || true
  echo "== relevant builder lines =="
  sed -n '260,340p' src/farkas/linopy/builder.py | cat -n
fi

echo "== test relevant lines =="
if [ -f tests/test_relational.py ]; then
  wc -l tests/test_relational.py
  sed -n '680,755p' tests/test_relational.py | cat -n
fi

echo "== search parameter/evaluator/coefficient/where/bound terms =="
rg -n "def .*eval|ParameterNode|coefficient|missing|bound|where|SPARSE_COEFFICIENT_MODEL|SPARSE_BOUND|sparse" -S .

Repository: FBumann/farkas

Length of output: 50370

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== builder evaluate/_eval helpers =="
sed -n '480,555p' src/farkas/linopy/builder.py | cat -n
sed -n '540,578p' src/farkas/linopy/builder.py | cat -n

echo "== semantics coefficients/missing behavior =="
wc -l src/farkas/linopy/semantics.py
sed -n '1,220p' src/farkas/linopy/semantics.py | cat -n

echo "== test harness and missing bound related tests =="
rg -n "def differential|class differential|SPARSE_COEFFICIENT_MODEL|missing.*bound|bound.*missing|NULL bounds|NaN|null|farkas\.relational|linopy\.extend" tests tests/src 2>/dev/null | sed -n '1,220p' || true
sed -n '1,120p' tests/test_relational.py | cat -n

echo "== relational bound null behavior references with context =="
sed -n '320,345p' src/farkas/relational/executor.py | cat -n
sed -n '648,670p' src/farkas/relational/executor.py | cat -n
sed -n '188,210p' src/farkas/relational/sql.py | cat -n

echo "== focused functional probe of current builder/semantics code via import if available? =="
python3 - <<'PY'
import sys
for m in ['src.farkas.linopy.semantics']:
    print("avail", m)
PY

Repository: FBumann/farkas

Length of output: 22174

---

Preserve missing values for parameters in non-coefficient positions.

ParameterNode currently applies coefficient(fillna(0.0)) from the evaluator, so sparse bound data silently becomes <= 0 on missing rows. This path is currently limited to constraint RHS/expression evaluation, but it already conflicts with the relational lane rule that missing bound parameters are NULL and raise. Keep coefficient filling out of semantics.coefficient and add zero-filling only at expression-coefficient join points; this also invalidates the sparse c omission in tests/test_relational.py, which should keep t=0 for the bound while testing missing bound errors separately.

📍 Affects 2 files

• src/farkas/linopy/builder.py#L310-L311 (this comment)
• tests/test_relational.py#L723-L731

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/farkas/linopy/builder.py` around lines 310 - 311, Update ParameterNode
handling in the builder’s evaluator to preserve missing parameter values by
removing coefficient-style zero filling from semantics.coefficient. Add
fillna(0.0) only at expression-coefficient join points where coefficient
evaluation requires it. In tests/test_relational.py lines 723-731, retain c with
t=0 for the bound case and add or adjust separate coverage for missing bound
parameters to verify they raise.

[coderabbitai]

📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift

Keep the new semantic helpers type-safe instead of widening to Any.

These exported helpers erase Pyrefly’s type checking at the compatibility boundary. Define narrow protocols for fillna() and to_linexpr() (or use supported public types) rather than using Any for every input and return value.

As per coding guidelines: “Keep Pyrefly strict type checking at zero errors; fix types rather than widening them.”

Also applies to: 64-64, 92-92

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/farkas/linopy/semantics.py` around lines 39 - 44, Replace the Any
annotations in the exported helpers coefficient, present, and vacated with
narrow protocols or supported public types describing the required fillna() and
to_linexpr() operations and their concrete return values. Preserve each helper’s
existing behavior while ensuring Pyrefly can type-check all inputs and outputs
without widening to Any.

Source: Coding guidelines