Allow creation of expandable external datasets

[ajelenak]

External HDF5 datasets can be expanded along their first dimension but cannot have chunked layout. This PR avoids setting chunked layout when creating a dataset if both maxshape and external keyword arguments are present.

Fixes #2396.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.53%. Comparing base (d051d24) to head (e506268).
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2398   +/-   ##
=======================================
  Coverage   89.53%   89.53%           
=======================================
  Files          17       17           
  Lines        2380     2380           
=======================================
  Hits         2131     2131           
  Misses        249      249           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[tacaswell]

Should there also be work upstream in libhdf5 to reject creating these broken data sets?

With this change can the user still explicitly ask for a broken dataset?

[takluyver]

I think HDF5 already rejects the combination of chunked & external storage (from #2394: ValueError: Unable to synchronously create dataset (external storage not supported with chunked layout)). But external & expandable (along axis 0) should work.

[tacaswell]

I think HDF5 already rejects the combination of chunked & external storage

But then how did the user end up with a hdf5 header which is invalid? Or am I confused by what is going on and we using this code on access?

[ajelenak]

Indeed, in other cases I tried libhdf5 raised errors.

[ajelenak]

That HDF5 file is not invalid. I got educated afterward about expanding external dataset along first dimension without the need for chunking. That's why I made this PR, to allow doing so from h5py.

[tacaswell]

That HDF5 file is not invalid.

Will this fix our reading as well then?

[takluyver]

The code involved here is only called when creating datasets, so I don't believe it will fix reading. @ajelenak spotted a separate issue while investigating #2394, which remains mysterious AFAIK.