Is it possible to train an unmodified version on MNC with a different dataset?

[javierjsa]

Hi,

Before attempting to modify MNC to work with a 2-categories dataset, I wondered if it would be possible to train an unmodified MNC with said dataset (which has of course been converted to PASCAL VOC 2012 format.)

I was expecting MNC to accept the data (even though it contains only examples of two categories) and just perform poorly. Unfortunately, it crashes:

(...)
Appending horizontally-flipped training examples...
0 / 3080
1000 / 3080
2000 / 3080
3000 / 3080
wrote gt roidb to /MNC/data/cache/voc_2012_train_gt_roidb.pkl
wrote gt flipped roidb to /MNC/data/cache/voc_2012_train_gt_roidb_flip.pkl
done
Preparing training data...
done
Loaded dataset `voc_2012_train` for training
Set proposal method: gt
Appending horizontally-flipped training examples...
voc_2012_train gt roidb loaded from /MNC/data/cache/voc_2012_train_gt_roidb.pkl
0 / 3080
Traceback (most recent call last):
  File "./tools/train_net.py", line 84, in <module>
    imdb, maskdb = attach_maskdb(args.imdb_name)
  File "/MNC/tools/../lib/db/maskdb.py", line 31, in attach_maskdb
    maskdbs = [get_maskdb(s) for s in imdb_names.split('+')]
  File "/MNC/tools/../lib/db/maskdb.py", line 22, in get_maskdb
    imdb.append_flipped_masks()
  File "/MNC/tools/../lib/datasets/pascal_voc_seg.py", line 135, in append_flipped_masks
    masks = self.maskdb[i]['gt_masks']
  File "/MNC/tools/../lib/datasets/pascal_voc.py", line 78, in maskdb
    self._maskdb = self.maskdb_handler()
  File "/MNC/tools/../lib/datasets/pascal_voc_seg.py", line 62, in gt_maskdb
    for index in xrange(num_image)]
  File "/MNC/tools/../lib/datasets/pascal_voc_seg.py", line 106, in _load_sbd_mask_annotations
    mask_max_x = max(gt_masks[i].shape[1] for i in xrange(len(gt_masks)))
ValueError: max() arg is an empty sequence

Any help to understand what the problem is would be appreciated.

Regards

[javierjsa]

Some images in my dataset contained no objects, only background. Thus, their associated mat files had empty fields. Removing such files solved the issue. However, it crashes again after loading VGG16 caffe model, right before starting the training:

File "./tools/train_net.py", line 93, in <module>
    pretrained_model=args.pretrained_model)
  File "/MNC/tools/../lib/caffeWrapper/SolverWrapper.py", line 61, in __init__
    self.solver.net.layers[0].set_roidb(roidb)
  File "/MNC/tools/../lib/pylayer/mnc_data_layer.py", line 63, in set_roidb
    self._shuffle_roidb_inds()
  File "/MNC/tools/../lib/pylayer/mnc_data_layer.py", line 81, in _shuffle_roidb_inds
    inds = np.reshape(inds, (-1, 2))
  File "/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 218, in reshape
    return reshape(newshape, order=order)
ValueError: total size of new array must be unchanged

[javierjsa]

Setting C.TRAIN.ASPECT_GROUPING=False solved the problem, yet Caffe does not validate the model. So, in short, it can't be done