Hard rule 3 says both lanes accept exactly the same language, and that is what makes the differential tests an oracle rather than a comparison of dialects. It is true of the language. It is not true of the data surface, and a user has no reason to draw that line.
The divergence
| Input |
compat (loader._coerce_to_dataarray) |
native (lowering.tidy_sources) |
parquet path (str / Path) |
❌ |
✅ |
dict |
✅ |
❌ |
np.ndarray / list |
✅ |
❌ |
| scalar |
✅ |
✅ |
pd.Series |
✅ |
✅ |
pd.DataFrame |
✅ (2-D, stacked) |
✅ (tidy (dims…, value)) |
xr.DataArray |
✅ |
✅ |
Note the DataFrame row is not really agreement either: compat expects a 2-D frame indexed by the two declared dims and stacks it; native expects a tidy frame with a value column.
What it looks like to a user
# works on compat, fails on native
compat.build('m.yaml', data={'cost': {'wind': 1.0, 'gas': 50.0}})
ly.solve('m.yaml', sources={'cost': {'wind': 1.0, 'gas': 50.0}})
# RelationalBuildError: parameter 'cost': cannot adapt dict to a tidy table
# works on native, fails on compat
ly.solve('m.yaml', sources={'cost': 'cost.parquet'})
compat.build('m.yaml', data={'cost': 'cost.parquet'})
# TypeError: Parameter 'cost': unsupported type 'str'.
So the same YAML, with the same meaning in both lanes, will not run in both lanes on the same data=. The differential tests do not catch this because they hand each lane inputs it happens to accept.
Why it is worth fixing rather than documenting
- It is the one remaining place where "one language, two lanes" is misleading in a way a user hits directly.
- The asymmetry is accidental, not designed:
loader.py grew for the eager path when that was the product, tidy_sources grew for the streaming path afterwards, and neither was reconciled.
- It is ~225 lines against ~60 doing overlapping work.
Sketch, not a decision
One coercion front-end producing the tidy (dims…, value) form, with the compat lane adapting tidy → xr.Dataset at its own boundary. The union of accepted types becomes the contract, and the differential harness should assert it — feed both lanes the same data= for every supported input type, which is what would have caught this.
Open questions worth settling first:
- Does the compat lane want parquet paths at all, given it is explicitly the not-for-scale lane? (If not, the contract is "native ⊇ compat", which is still a statable rule — just not the current one.)
dict and list are conveniences that only make sense for small hand-written models. Are they worth carrying into the streaming path, or worth dropping from compat?
- 2-D
DataFrame handling is genuinely different in kind, not just in coverage. Unifying it means picking one meaning.
Found while cutting back accumulated surface (#56, #59); deliberately left out of both because it is a design decision rather than tidying.
Hard rule 3 says both lanes accept exactly the same language, and that is what makes the differential tests an oracle rather than a comparison of dialects. It is true of the language. It is not true of the data surface, and a user has no reason to draw that line.
The divergence
loader._coerce_to_dataarray)lowering.tidy_sources)str/Path)dictnp.ndarray/listpd.Seriespd.DataFrame(dims…, value))xr.DataArrayNote the
DataFramerow is not really agreement either: compat expects a 2-D frame indexed by the two declared dims and stacks it; native expects a tidy frame with avaluecolumn.What it looks like to a user
So the same YAML, with the same meaning in both lanes, will not run in both lanes on the same
data=. The differential tests do not catch this because they hand each lane inputs it happens to accept.Why it is worth fixing rather than documenting
loader.pygrew for the eager path when that was the product,tidy_sourcesgrew for the streaming path afterwards, and neither was reconciled.Sketch, not a decision
One coercion front-end producing the tidy
(dims…, value)form, with the compat lane adapting tidy →xr.Datasetat its own boundary. The union of accepted types becomes the contract, and the differential harness should assert it — feed both lanes the samedata=for every supported input type, which is what would have caught this.Open questions worth settling first:
dictandlistare conveniences that only make sense for small hand-written models. Are they worth carrying into the streaming path, or worth dropping from compat?DataFramehandling is genuinely different in kind, not just in coverage. Unifying it means picking one meaning.Found while cutting back accumulated surface (#56, #59); deliberately left out of both because it is a design decision rather than tidying.