HDF5 crashed when using layer HDF5_OUTPUT

[tartavull]

it works on the beginning but it crashes when rewriting to it
I0114 19:41:35.961170 18670 hdf5_output_layer.cpp:33] Saving HDF5 file/usr/people/me/caffe/examples/test/test_prediction.h5
I0114 19:41:35.962885 18670 hdf5_output_layer.cpp:38] Successfully saved 100 rows
I0114 19:41:35.986912 18670 hdf5_output_layer.cpp:33] Saving HDF5 file/usr/people/me/caffe/examples/test/test_prediction.h5
HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 140309532535360:
#000: ../../../src/H5D.c line 170 in H5Dcreate2(): unable to create dataset
major: Dataset
minor: Unable to initialize object
#1: ../../../src/H5Dint.c line 439 in H5D__create_named(): unable to create and link to dataset
major: Dataset
minor: Unable to initialize object
#2: ../../../src/H5L.c line 1638 in H5L_link_object(): unable to create new link to object
major: Links
minor: Unable to initialize object
#3: ../../../src/H5L.c line 1882 in H5L_create_real(): can't insert link
major: Symbol table
minor: Unable to insert object
#4: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
major: Symbol table
minor: Object not found
#5: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
major: Symbol table
minor: Callback failed
#6: ../../../src/H5L.c line 1674 in H5L_link_cb(): name already exists
major: Symbol table
minor: Object already exists
F0114 19:41:35.987098 18670 io.cpp:178] Check failed: status >= 0 (-1 vs. 0) Failed to make float dataset data
*** Check failure stack trace: ***
@ 0x7f9c5635bdaa (unknown)
@ 0x7f9c5635bce4 (unknown)
@ 0x7f9c5635b6e6 (unknown)
@ 0x7f9c5635e687 (unknown)
@ 0x4a0a41 caffe::hdf5_save_nd_dataset<>()
@ 0x4feee8 caffe::HDF5OutputLayer<>::SaveBlobs()
@ 0x5242d9 caffe::HDF5OutputLayer<>::Forward_gpu()
@ 0x45a6ab caffe::Net<>::ForwardFromTo()
@ 0x45aad7 caffe::Net<>::ForwardPrefilled()
@ 0x473b13 caffe::Solver<>::Test()
@ 0x4743e6 caffe::Solver<>::TestAll()
@ 0x47b1d5 caffe::Solver<>::Solve()
@ 0x415172 train()
@ 0x40fe41 main
@ 0x7f9c51de6ec5 (unknown)
@ 0x413ba7 (unknown)
@ (nil) (unknown)

[mingtop]

I got the same error :
#000: ../../../src/H5D.c line 170 in H5Dcreate2(): unable to create dataset

In my opion, caffe only save one circle's output in H5

[mingtop]

why no one can explain, it's a bug in caffe ?

[chenxistephen]

Got the same issue. I think caffe creates a new HDF5 dataset whenever it writes using HDF5_output layer. It's a bug that should be fixed

[normanyu]

I have the same issue. @chenxistephen do you know of any quick fixes?

[shelhamer]

Sorry, I've never used this layer but if someone with a use for it can send
a PR with a fix it would be welcome (and it seems there is enough interest
in it help others).

On Tue, Feb 17, 2015 at 2:23 PM, normanyu notifications@github.com wrote:

I have the same issue. @chenxistephen https://github.com/chenxistephen
do you know of any quick fixes?

—
Reply to this email directly or view it on GitHub
#1726 (comment).

[chenxistephen]

I think just changing the "H5LTmake_dataset_float" in "hdf5_save_nd_dataset" in "io.cpp" into writing a dataset if existed would fix. Sorry not familiar with low level HDF API either.

[prclibo]

Looks like H5LTmake_dataset_double is trying to create datasets with same name when processing for more than one batch. I noticed that #1392 fixes the problem as batches are named as different datasets.

[xucong-zhang]

@prclibo
Thanks for the link, it indeed solved the problem!

[lood339]

@XucongZhang
I have the same problem with writing hdf5 files. I realize that this link #1392 can solve the problem. However, I do not know how to merge the pull into my caffe (master). Can you help me some git commands so that the pull 1392 will merge to my caffe?
Thanks very much.

[xucong-zhang]

@lood339
Hi, you need to merge it into current version of caffe by hand. There is no simple command to do the trick.

[lood339]

Got it. Thanks very much.
On Feb 2, 2016 1:15 AM, "Xucong Zhang" notifications@github.com wrote:

@lood339 https://github.com/lood339
Hi, you need to merge it into current version of caffe by hand. There is
no simple command to do the trick.

—
Reply to this email directly or view it on GitHub
#1726 (comment).

[duygusar]

@prclibo @shelhamer I think you might have given some insight on the reason why I might be having trouble (while input though sounds similar), but I have the latest caffe so I don't know why I still have a bug and how to fix it.

Here is my problem: #6122