-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
What is your issue?
From xarray 2025.9.1 onwards it seems that netcdf4 is not the default engine anymore, but h5netcdf. I thought the PR was #10755, but this was merged 1 day after the 2025.9.1 release. Either way, with this xarray version I see that the default engine is different. I am running into troubles with opening netcdf files from an opendap server. When not setting the engine, there are warnings for h5netcdf and scipy, so I deliberately set it in this example:
import xarray as xr
file_nc_bathy = r'https://opendap.deltares.nl/thredds/dodsC/opendap/deltares/Delft3D/netcdf_example_files/GEBCO_2022/GEBCO_2022_coarsefac08.nc'
data_bathy = xr.open_dataset(file_nc_bathy, engine='h5netcdf')
This raises "FileNotFoundError: https://opendap.deltares.nl/thredds/dodsC/opendap/deltares/Delft3D/netcdf_example_files/GEBCO_2022/GEBCO_2022_coarsefac08.nc"
. When I use engine='netcdf4'
. If aiohttp is not installed I get "ImportError: HTTPFileSystem requires "requests" and "aiohttp" to be installed"
(but installing it still gives the FileNotFoundError
).
I am not sure whether this is a bug and if so, in which package exactly. I would expect that this new default engine, one should also be able to open files on opendap. Although, it seems from pytroll/satpy#786 (comment) that this might not be the case.
The thing is that with the new default engine, users get confusing errors like ImportError
and FileNotFoundError
. Or if they do not manually set the engine (most of the cases), they get warnings:
C:\Users\veenstra\AppData\Local\miniforge3\envs\dfm_tools_env\Lib\site-packages\xarray\backends\api.py:577: RuntimeWarning: 'h5netcdf' fails while guessing
engine = plugins.guess_engine(filename_or_obj)
C:\Users\veenstra\AppData\Local\miniforge3\envs\dfm_tools_env\Lib\site-packages\xarray\backends\api.py:577: RuntimeWarning: 'scipy' fails while guessing
engine = plugins.guess_engine(filename_or_obj)
Furthermore, in my github tests, I get the ImportError
without explicitly setting the engine: Deltares/dfm_tools#1265
Proposed solution
In #10755 there is an exception built in for nczarr:
nczarr_mode = isinstance(path_or_file, str) and path_or_file.endswith(
"#mode=nczarr"
)
if nczarr_mode:
candidates[:] = ["netcdf4"]
Would it also be possible to build this in for opendap, so users do not get potentially confusing errors/warnings?
Additional request
Furthermore, I did notice less memory consumption but much more time consumption with h5netcdf in the past: Deltares/dfm_tools#484. I had hoped that the h5netcdf performance issues (h5netcdf/h5netcdf#195 and h5netcdf/h5netcdf#251) would have been resolved before h5netcdf was chosen as the default xarray engine. Would it still be possible to do so?