Skip to content

new default h5netcdf engine does not support opendap #10810

@veenstrajelmer

Description

@veenstrajelmer

What is your issue?

From xarray 2025.9.1 onwards it seems that netcdf4 is not the default engine anymore, but h5netcdf. I thought the PR was #10755, but this was merged 1 day after the 2025.9.1 release. Either way, with this xarray version I see that the default engine is different. I am running into troubles with opening netcdf files from an opendap server. When not setting the engine, there are warnings for h5netcdf and scipy, so I deliberately set it in this example:

import xarray as xr
file_nc_bathy = r'https://opendap.deltares.nl/thredds/dodsC/opendap/deltares/Delft3D/netcdf_example_files/GEBCO_2022/GEBCO_2022_coarsefac08.nc'
data_bathy = xr.open_dataset(file_nc_bathy, engine='h5netcdf')

This raises "FileNotFoundError: https://opendap.deltares.nl/thredds/dodsC/opendap/deltares/Delft3D/netcdf_example_files/GEBCO_2022/GEBCO_2022_coarsefac08.nc". When I use engine='netcdf4'. If aiohttp is not installed I get "ImportError: HTTPFileSystem requires "requests" and "aiohttp" to be installed" (but installing it still gives the FileNotFoundError).

I am not sure whether this is a bug and if so, in which package exactly. I would expect that this new default engine, one should also be able to open files on opendap. Although, it seems from pytroll/satpy#786 (comment) that this might not be the case.

The thing is that with the new default engine, users get confusing errors like ImportError and FileNotFoundError. Or if they do not manually set the engine (most of the cases), they get warnings:

C:\Users\veenstra\AppData\Local\miniforge3\envs\dfm_tools_env\Lib\site-packages\xarray\backends\api.py:577: RuntimeWarning: 'h5netcdf' fails while guessing
  engine = plugins.guess_engine(filename_or_obj)
C:\Users\veenstra\AppData\Local\miniforge3\envs\dfm_tools_env\Lib\site-packages\xarray\backends\api.py:577: RuntimeWarning: 'scipy' fails while guessing
  engine = plugins.guess_engine(filename_or_obj)

Furthermore, in my github tests, I get the ImportError without explicitly setting the engine: Deltares/dfm_tools#1265

Proposed solution

In #10755 there is an exception built in for nczarr:

    nczarr_mode = isinstance(path_or_file, str) and path_or_file.endswith(
        "#mode=nczarr"
    )
    if nczarr_mode:
        candidates[:] = ["netcdf4"]

Would it also be possible to build this in for opendap, so users do not get potentially confusing errors/warnings?

Additional request

Furthermore, I did notice less memory consumption but much more time consumption with h5netcdf in the past: Deltares/dfm_tools#484. I had hoped that the h5netcdf performance issues (h5netcdf/h5netcdf#195 and h5netcdf/h5netcdf#251) would have been resolved before h5netcdf was chosen as the default xarray engine. Would it still be possible to do so?

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs triageIssue that has not been reviewed by xarray team member

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions