New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue reading SVHN dataset on Windows #6017

Closed
AlexDBlack opened this Issue Jul 31, 2018 · 4 comments

Comments

Projects
None yet
2 participants
@AlexDBlack
Copy link
Member

AlexDBlack commented Jul 31, 2018

I've run run the YOLO house number detection example recently on Linux (with snapshots), and it ran fine.

Now, when I try to run it on Windows, I get the following on both the train and the test sets:

Exception in thread "main" java.lang.RuntimeException: H5Dread failed
	at org.bytedeco.javacpp.hdf5$DataSet.read(Native Method)
	at org.datavec.image.recordreader.objdetect.impl.SvhnLabelProvider.<init>(SvhnLabelProvider.java:76)
	at org.deeplearning4j.examples.convolution.objectdetection.HouseNumberDetection.main(HouseNumberDetection.java:96)
HDF5-DIAG: Error detected in HDF5 (1.10.2) thread 0:
  #000: C:\projects\javacpp-presets\hdf5\cppbuild\windows-x86_64\hdf5-1.10.2\src\H5Dio.c line 216 in H5Dread(): can't read data
    major: Dataset
    minor: Read failed
  #001: C:\projects\javacpp-presets\hdf5\cppbuild\windows-x86_64\hdf5-1.10.2\src\H5Dio.c line 587 in H5D__read(): can't read data
    major: Dataset
    minor: Read failed
  #002: C:\projects\javacpp-presets\hdf5\cppbuild\windows-x86_64\hdf5-1.10.2\src\H5Dchunk.c line 2304 in H5D__chunk_read(): unable to read raw data chunk
    major: Low-level I/O
    minor: Read failed
  #003: C:\projects\javacpp-presets\hdf5\cppbuild\windows-x86_64\hdf5-1.10.2\src\H5Dchunk.c line 3659 in H5D__chunk_lock(): data pipeline read failed
    major: Data filters
    minor: Filter operation failed
  #004: C:\projects\javacpp-presets\hdf5\cppbuild\windows-x86_64\hdf5-1.10.2\src\H5Z.c line 1254 in H5Z_pipeline(): required filter 'deflate' is not registered
    major: Data filters
    minor: Read failed
  #005: C:\projects\javacpp-presets\hdf5\cppbuild\windows-x86_64\hdf5-1.10.2\src\H5PLint.c line 270 in H5PL_load(): search in path table failed
    major: Plugin for dynamically loaded library
    minor: Can't get value
  #006: C:\projects\javacpp-presets\hdf5\cppbuild\windows-x86_64\hdf5-1.10.2\src\H5PLpath.c line 604 in H5PL__find_plugin_in_path_table(): search in path C:\ProgramData\hdf5\lib\plugin encountered an error
    major: Plugin for dynamically loaded library
    minor: Can't get value
  #007: C:\projects\javacpp-presets\hdf5\cppbuild\windows-x86_64\hdf5-1.10.2\src\H5PLpath.c line 734 in H5PL__find_plugin_in_path(): can't open directory
    major: Plugin for dynamically loaded library
    minor: Can't open directory or file

The path looks correct, and changing the separators to all / or \ doesn't make a difference...

Edit: seems to me that the problem isn't the actual dataset at all - rather, it's an issue with the HDF5 library not being able to scan for plugins??
H5PL__find_plugin_in_path(): can't open directory

https://support.hdfgroup.org/ftp///HDF5/prev-releases/hdf5-1.10/hdf5-1.10.2/src/unpacked/src/H5PLpath.c

/*-------------------------------------------------------------------------
 * Function:    H5PL__find_plugin_in_path
 *
 * Purpose:     Given a path, this function opens the directory and envokes
 *              another function to go through all files to find the right
 *              plugin library. Two function definitions are for Unix and
 *              Windows.
@saudet

This comment has been minimized.

Copy link
Member

saudet commented Jul 31, 2018

Hum, looks like the compression algorithm isn't getting built on Windows...

@AlexDBlack

This comment has been minimized.

Copy link
Member

AlexDBlack commented Jul 31, 2018

So, yes it's plugin related...

search in path C:\ProgramData\hdf5\lib\plugin encountered an error

Creating this directory doesn't solve the problem.
Nor does overriding the HDF5_PLUGIN_PATH environment variable :/

  #006: C:\projects\javacpp-presets\hdf5\cppbuild\windows-x86_64\hdf5-1.10.2\src\H5PLpath.c line 604 in H5PL__find_plugin_in_path_table(): search in path C:\Temp\HDF5 encountered an error
    major: Plugin for dynamically loaded library
    minor: Can't get value
  #007: C:\projects\javacpp-presets\hdf5\cppbuild\windows-x86_64\hdf5-1.10.2\src\H5PLpath.c line 734 in H5PL__find_plugin_in_path(): can't open directory
    major: Plugin for dynamically loaded library
    minor: Can't open directory or file

@saudet saudet self-assigned this Jul 31, 2018

saudet added a commit to bytedeco/javacpp-presets that referenced this issue Aug 1, 2018

@saudet

This comment has been minimized.

Copy link
Member

saudet commented Oct 16, 2018

That's now fixed with the JavaCPP Presets for HDF5 1.10.3-1.4.3.

@saudet saudet closed this Oct 16, 2018

@lock

This comment has been minimized.

Copy link

lock bot commented Nov 15, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Nov 15, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.