Seeking help on using pretrained models for inferencing

[stephenjlee]

Dear MatConvNet FCN community,

I'm new to semantic segmentation and was hoping to test the pretrained models posted on the MatConvNet website. I'm simply trying to view inferences made by the model "pascal-fcn32s-dag.mat" on peppers.png.

I'm trying two approaches and getting stuck in both.

1. In fcnTest.m, redirecting opts.modelPath from 'data/fcn32-voc11/net-epoch-50.mat' to the pretrained model. I get errors regarding the structure of the pretrained model not matching what is expected.
2. loading the pretrained model in a test script using the DagNN wrapper and running net.eval({'data', im_}). It runs and I get a 384x512x21 value corresponding to 'upscore,' but I'm not sure how to correctly interpret/print the segmentation.

Any help would be greatly appreciated!

Thank you,
Stephen

[stephenjlee]

Ah I see what's going on now. I didn't before realize how to correctly call fcnTest using the pretrained models until I started looking at fcnTestModelZoo. For future users who may initially be similarly confused, I'm now setting opts.modelFamily to 'ModelZoo' and it works now.

Great implementation!

[ectg]

Hi Stephen,

Were you able to do semantic region(not class based) segmentation using this code? I looked at fcnTest.m, vocSetup seems to be setting up around 20 predefined classes for the testing, but I was wondering if it would label semantically similar regions given a test image like peppers.png. Have you got any suggestions?

Thanks!

[stephenjlee]

Hi ectg. I'm not sure I understand your question. I was able to retrain the network to detect different classes. You can retrain/fine tune the models to look for anything you want.

What do you mean by "(not class based)" - are you looking to do just normal segmentation?

[ectg]

Hi Stephen, Thanks for the reply. Yes I was looking to do normal segmentation to start off with, but I saw yesterday that the pretrained models like pascal-fcn8s-dag.mat aren't suited for that. I would instead like to do a 2 class segmentation of 'person' vs the rest, but I have't figured out how to see the list of classes used to train that model, the result I get from the evaluation is of dimension 500x500x21, 21 classes I assume, one of which is 'person'.

I am new to deep learning, just started last week, hence the noob questions.

[ectg]

Hi, So I was able to do this segmentation, but have some trouble extracting features from the CNN for transfer learning.

I am using imagenet-caffe-alex.mat for feature extraction from the test region. The result from layer 'fc7' after I run my test image of size 500x500x3 is of dimension 10x10x4096, here is the code.

res = vl_simplenn(alexNet, im_) ;
layer_number = 18; %'fc7'
featureVector = res(layer_number).x;

Shouldn't the featureVector be a 4096x1 vector? Do you know what is the significance of the 10x10 patch?

[roshambo919]

Hi Stephen et al.,

I am also setting up a segmentation FCN based on the pascal model. I was having the same kinds of questions as you initially whereby I have run the network and have the 'upscore' variable stored as a 21 layer map of values.

Did you figure out how to analyze this upscore variable? As in, how did you go from this variable to the ultimate segmented image?

Any kind of help on this would be appreciated!

Thanks