Skip to content

The two lanes accept different data inputs, though they accept the same language #60

Description

@FBumann

Hard rule 3 says both lanes accept exactly the same language, and that is what makes the differential tests an oracle rather than a comparison of dialects. It is true of the language. It is not true of the data surface, and a user has no reason to draw that line.

The divergence

Input compat (loader._coerce_to_dataarray) native (lowering.tidy_sources)
parquet path (str / Path)
dict
np.ndarray / list
scalar
pd.Series
pd.DataFrame ✅ (2-D, stacked) ✅ (tidy (dims…, value))
xr.DataArray

Note the DataFrame row is not really agreement either: compat expects a 2-D frame indexed by the two declared dims and stacks it; native expects a tidy frame with a value column.

What it looks like to a user

# works on compat, fails on native
compat.build('m.yaml', data={'cost': {'wind': 1.0, 'gas': 50.0}})
ly.solve('m.yaml', sources={'cost': {'wind': 1.0, 'gas': 50.0}})
#   RelationalBuildError: parameter 'cost': cannot adapt dict to a tidy table

# works on native, fails on compat
ly.solve('m.yaml', sources={'cost': 'cost.parquet'})
compat.build('m.yaml', data={'cost': 'cost.parquet'})
#   TypeError: Parameter 'cost': unsupported type 'str'.

So the same YAML, with the same meaning in both lanes, will not run in both lanes on the same data=. The differential tests do not catch this because they hand each lane inputs it happens to accept.

Why it is worth fixing rather than documenting

  • It is the one remaining place where "one language, two lanes" is misleading in a way a user hits directly.
  • The asymmetry is accidental, not designed: loader.py grew for the eager path when that was the product, tidy_sources grew for the streaming path afterwards, and neither was reconciled.
  • It is ~225 lines against ~60 doing overlapping work.

Sketch, not a decision

One coercion front-end producing the tidy (dims…, value) form, with the compat lane adapting tidy → xr.Dataset at its own boundary. The union of accepted types becomes the contract, and the differential harness should assert it — feed both lanes the same data= for every supported input type, which is what would have caught this.

Open questions worth settling first:

  • Does the compat lane want parquet paths at all, given it is explicitly the not-for-scale lane? (If not, the contract is "native ⊇ compat", which is still a statable rule — just not the current one.)
  • dict and list are conveniences that only make sense for small hand-written models. Are they worth carrying into the streaming path, or worth dropping from compat?
  • 2-D DataFrame handling is genuinely different in kind, not just in coverage. Unifying it means picking one meaning.

Found while cutting back accumulated surface (#56, #59); deliberately left out of both because it is a design decision rather than tidying.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:apiNative entry points and CLIarea:compatOracle lane and linopy interopdecisionA question to be answered, not work to be done; closes by resolution

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions