Repository navigation
What's Changed
The on-demand loader prm.dataset gets a lazy, Dask-backed successor built on xarray-binfile 0.3, and starts its deprecation. Existing code keeps working, with a warning.
Added
-
Lazy, Dask-backed IO on xarray-binfile; deprecate prm.dataset by @fschuch in #142
prm.open_dataset()opens every snapshot of a simulation as one lazyxarray.Dataset. Nothing is read until values are needed, one Dask task per file by default, with velocity components stacked oni(ufromux,uy,uz) and scalar fractions onn(phifromphi1,phi2, ...).stack_namesreassembles user-defined fields too, for example{"i": {"u", "vort"}}; stack=False` returns the variables as on disk.prm.write_dataset()writes arrays and datasets back to raw binaries with the familiarfile_name/file_prefixrules. Any array carryingiornis split into one file per component, whatever its name;tgives one file per snapshot. Writes are atomic per file and computed one file at a time.Xcompact3dConvention(xcompact3d_toolbox.binfile): the description of the files (Fortran byte order, naming,t = step * dt * ioutput, static fields, sub-folders declared per case withfolders=), reusable withxarray.open_mfdatasetand.binary_engine.to_filedirectly. The dtype is chosen per call;param["mytype"]is only the default.engine="xcompact3d"xarray backend:xr.open_mfdataset("case/data/*.bin", engine="xcompact3d", prm=prm). Withoutprm=, the single*.i3dor*.prmnext to the data folder is loaded automatically.prm.default_data_path, and an explicitdata_pathargument oninit_epsi,init_datasetandgene_epsi_3d, which now write through the new API.- Documentation: a migration guide mapping every
prm.datasetcall to its replacement, a "Lazy loading with xarray-binfile" section in the IO tutorial, and API reference pages for the new modules.
Deprecated
prm.datasetemits aFutureWarningand will be removed in 2.0. The lazy API is independent of it: configure one or the other, never both.write_xdmfandload_wind_turbine_datastay onprm.datasetuntil they get a new home.
Changed
- xarray-binfile 0.3 is a required dependency.
- Output names written by the new API are letters, digits and underscores, optionally under sub-folders; the separator, counter and extension come from the convention, so
file_prefix="w3-mean"is rejected with a clear message (the old writer accepted it). - Sandbox inflow planes (
bxx1,byphi1, ...) are written by the new API but never listed byopen_dataset; read them withxr.open_dataset(path, engine="xcompact3d", prm=prm, drop_coords=...).
Internals
- chore(deps): bump the actions group with 2 updates by @dependabot[bot] in #131
- chore(deps): bump actions/setup-python from 6.2.0 to 6.3.0 in the actions group by @dependabot[bot] in #134
- chore(deps): bump the actions group across 1 directory with 5 updates by @dependabot[bot] in #140
Other Changes
- pre-commit autoupdate by @pre-commit-ci[bot] in #133
- pre-commit autoupdate by @pre-commit-ci[bot] in #135
- pre-commit autoupdate by @pre-commit-ci[bot] in #141
Full Changelog: v1.4.0...v1.5.0