Error using HDF5 data for training

[sohail1436]

Hello There,
I have been trying to train the caffenet model using HDF5 data I used the prototxt files from ~/../caffe/examples/hdf5_classification. But I get the following error
I0219 19:06:43.038563 5773 caffe.cpp:103] Use CPU.
I0219 19:06:43.052496 5773 caffe.cpp:107] Starting Optimization
I0219 19:06:43.052989 5773 solver.cpp:32] Initializing solver from parameters:
test_iter: 10
test_interval: 10
base_lr: 0.01
display: 10
max_iter: 20
lr_policy: "step"
gamma: 0.1
momentum: 0.9
weight_decay: 0.0005
stepsize: 5
snapshot: 10
snapshot_prefix: "examples/hdf5_classification/train"
solver_mode: CPU
net: "examples/hdf5_classification/train_val2.prototxt"
I0219 19:06:43.055266 5773 solver.cpp:67] Creating training net from net file: examples/hdf5_classification/train_val2.prototxt
I0219 19:06:43.056095 5773 net.cpp:275] The NetState phase (0) differed from the phase (1) specified by a rule in layer data
I0219 19:06:43.056354 5773 net.cpp:275] The NetState phase (0) differed from the phase (1) specified by a rule in layer accuracy
I0219 19:06:43.056669 5773 net.cpp:39] Initializing net from parameters:
name: "LogisticRegressionNet"
layers {
top: "data"
top: "label"
name: "data"
type: HDF5_DATA
hdf5_data_param {
source: "examples/hdf5_classification/data/tr.txt"
batch_size: 1
}
include {
phase: TRAIN
}
}
layers {
bottom: "data"
top: "fc1"
name: "fc1"
type: INNER_PRODUCT
blobs_lr: 1
blobs_lr: 2
weight_decay: 1
weight_decay: 0
inner_product_param {
num_output: 40
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layers {
bottom: "fc1"
top: "fc1"
name: "relu1"
type: RELU
}
layers {
bottom: "fc1"
top: "fc2"
name: "fc2"
type: INNER_PRODUCT
blobs_lr: 1
blobs_lr: 2
weight_decay: 1
weight_decay: 0
inner_product_param {
num_output: 2
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layers {
bottom: "fc2"
bottom: "label"
top: "loss"
name: "loss"
type: SOFTMAX_LOSS
}
state {
phase: TRAIN
}
I0219 19:06:43.061589 5773 net.cpp:67] Creating Layer data
I0219 19:06:43.061820 5773 net.cpp:356] data -> data
I0219 19:06:43.062067 5773 net.cpp:356] data -> label
I0219 19:06:43.062321 5773 net.cpp:96] Setting up data
I0219 19:06:43.062525 5773 hdf5_data_layer.cpp:57] Loading filename from examples/hdf5_classification/data/tr.txt
I0219 19:06:43.062893 5773 hdf5_data_layer.cpp:69] Number of files: 1
I0219 19:06:43.063103 5773 hdf5_data_layer.cpp:29] Loading HDF5 file/home/sohail/caffe/examples/hdf5_classification/data/new.hd5
HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 3008321728:
#000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
major: Dataset
minor: Object not found
#1: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
major: Symbol table
minor: Object not found
#2: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
major: Symbol table
minor: Object not found
#3: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
major: Symbol table
minor: Callback failed
#4: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'label' doesn't exist
major: Symbol table
minor: Object not found
F0219 19:06:43.255592 5773 io.cpp:131] Check failed: status >= 0 (-1 vs. 0) Failed to get dataset ndims for label
*** Check failure stack trace: ***
Aborted (core dumped)

Can someone help me regarding this.....
Cheers

[rohrbach]

Did you check that the HD5 file exists (i.e. that the path is correct) and that it contains the following two data structures: "data" and "label"?

However, please ask usage questions on caffe-users -- the issues tracker is primarily for Caffe development discussion. Thanks!

[sohail1436]

@rohrbach Yes my file wasn't having labels added labels data structure and the problem was solved
Thanks for the help

[Nateliu712]

@sohail1436 I meet the same question. Could you help solve this problem?

[duygusar]

Does it have to have the name "data" and "label"? I have three tops, one imagedata and two labels with different naming conventions.