Reading hyperslabs fails

[joshua-gould]

Example:
Prerequisites: Download hdf5 file from http://pklab.med.harvard.edu/velocyto/hgForebrainGlut/hgForebrainGlut.loom

var hdf5 = require('hdf5').hdf5;
var h5lt = require('hdf5').h5lt;
var Access = require('hdf5/lib/globals').Access;
var file = new hdf5.File('hgForebrainGlut.loom', Access.ACC_RDONLY);
var dim = file.getDatasetDimensions('matrix');
for (var i = 0; i < dim[0]; i++) {
  var buffer = h5lt.readDatasetAsBuffer(file.id, 'matrix', {
    start: [i, 0],
    stride: [1, 1],
    count: [1, dim[1]]
  });
}

Error:
HDF5-DIAG: Error detected in HDF5 (1.8.20) thread 0:
#000: H5S.c line 1382 in H5Screate_simple(): maxdims is smaller than dims
major: Invalid arguments to routine
minor: Bad value
HDF5-DIAG: Error detected in HDF5 (1.8.20) thread 0:
#000: H5Dio.c line 147 in H5Dread(): not a data space
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.8.20) thread 0:
#000: H5S.c line 390 in H5Sclose(): not a dataspace
major: Invalid arguments to routine
minor: Inappropriate type
SyntaxError: failed to read dataset

[rimmartin]

Hi, I'll test with the example you provided and let you know. I'm able to catch up in the next couple of evenings(was away for the weekend)

[joshua-gould]

Thanks.

[rimmartin]

This is a compressed dataset. I've been working on getting compression capability.

I haven't tried a compressed dataset with hyperslabbing yet. Studying this some more...

[rimmartin]

hdf5 forum tells me this is doable.
https://forum.hdfgroup.org/t/are-hyperslabs-and-compression-compatible/860

Looking for a c example. it may have to do with access property list for the reading step. Looking...

[joshua-gould]

Awesome! Thanks so much.

…

On Wed, Feb 28, 2018 at 1:00 PM rimmartin ***@***.***> wrote: hdf5 forum tells me this is doable. https://forum.hdfgroup.org/t/are-hyperslabs-and-compression-compatible/860 Looking for a c example. it may have to do with access property list for the reading step. Looking... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#55 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AE_pnzT_pAWl7B7lthz5C2LD2z03MqKlks5tZZRHgaJpZM4SSpCQ> .

[joshua-gould]

Any updates on how to fix this issue? Thanks.

[rimmartin]

I'm getting back to things. Had a software release at work that consumed a lot of time

I'll make a small c/c++ app to test this out a bit; it really should be no different

[NINI1988]

File h5_lt.hpp Line(1417)
Maybe this memspace_id = H5Screate_simple(rank, count.get(), NULL);

Or
const hsize_t maxsize = H5S_UNLIMITED;
should be
std::unique_ptr<hsize_t[]> maxsize(new hsize_t[rank]);

Maybe the dimensions are multiple time wrong in the code?

[rimmartin]

in some versions of hdf5 H5S_UNLIMITED didn't work. You're making my recaller kick in:-) Will test some things looking at your other fix for dimensions

[rimmartin]

I think that was it! Must reveal itself when it is a compressed h5. Survey for more places an array should be used

[rimmartin]

All other H5Screate_simple's have the maximum_dims NULL. Will need to see if any should be filed in to allow H5S_UNLIMITED

[rimmartin]

@joshua-gould I committed changes after applying @NINI1988 's fix to this one as well.

[joshua-gould]

It works now! Thanks.