Error in caffe.Net/reshape (line 171) on a different dataset

[Isadorable]

Hello,
I'm trying to train a Faster-RNN (ZF) with a toy dataset I've built following accurately the structure of Pascal VOC 2007 (just to make sure it can run it properly with alternative datasets before building a "more serious" one).
Unfortunately when I try to train the net with my data I always get the same error message:

Error using caffe_
glog check error, please check log and clear mex
Error in caffe.Net/reshape (line 171)
caffe_('net_reshape', self.hNet_self);
Error in caffe.Net/reshape_as_input (line 186)
self.reshape();
Error in fast_rcnn_train>check_gpu_memory (line 213)
caffe_solver.net.reshape_as_input(net_inputs);
Error in fast_rcnn_train (line 90)
check_gpu_memory(conf, caffe_solver, num_classes, opts.do_val);
Error in script_fast_rcnn_VOC2007_ZF (line 36)
opts.fast_rcnn_model = fast_rcnn_train(conf, dataset.imdb_train, dataset.roidb_train, ...

after

Preparing training data...Done.
Preparing validation data...Done.

Caffe log file:

F1206 11:26:19.150897 8364 smooth_L1_loss_layer.cpp:32] Check failed: bottom[0]->channels() == bottom[1]->channels() (84 vs. 24)

I'd like to clarify that using the original Pascal VOC 2007 dataset I can train the net without any error message with both script_fast_rcnn_VOC2007_ZF and script_faster_rcnn_VOC2007_ZF. Same for script_faster_rcnn_demo in GPU mode.

I've noticed that other people are experiencing similar problems (e.g. #101) but in different circumstances.

I tested the code with both a GTX 780 and a GTX 970 that should be enough for this purpose.

My dataset consists of 44 jpg images (224x224x3) and four classes organised in Annotations, ImageSets and JPEGImages folders similarly to Pascal VOC 2007.
I'm using MATLAB R2016b and CUDA 8.

Does anyone have any tip? Is there, perhaps, a minimum size for the dataset?

[Isadorable]

So, if anyone happens to have the same problem here is the solution : Link