Evaluating FasterRCNN error

[mortengryning]

Hi.

I'm testing the new FasterRCNN example. I Use the CNTK GPU Build 2.1 and the CNTK.GPU nuget package in C#.

The training runs fine and archives a very nice Mean AP using the python scripts:

The test images also looks correct and everything besides the water is recognized which the output also indicate.

Now, when I try to load the model using the below code i get the following error:

I'm not sure what to do about the error: Please make sure to specify a valid UDF deserializer.".

If I train a model using the FastRCNN (using same installation and evaluation library) and evaluate it using the exact same code it works perfectly fine.

How do I fix this? FasterRCNN is much preferred because I don't have to do the ROI calculations.

[pkranen]

Faster RCNN uses a few Python UDFs (user defined functions) for training and one of them (the ProposalLayer) also for inference. You can think of these UDFs as new computation nodes. They are not yet available in native code at the moment, but we are working on a C++ version of the ProposalLayer to allow inference from C# etc. Stay tuned! (If you want to try early bits you can checkout alrezni/proposal_layer).

[mortengryning]

Hi.

Thank you for your answer.

If I use the branch you linked (alrezni/proposal_layer), will I be able to evaluate using the c# evaluation library or do I have to build/code some things first?

[pkranen]

We've tried yesterday to evaluate FasterRCNN from C# using the native proposal layer mentioned above. It required a few minor changes (due to FreeDimension), but then it worked. We are working on a clean solution for this and will post it once the changes are in the master branch. Thanks.

[mortengryning]

Great :-). I'm looking forward to testing it once it's in the master branch.

[Dozer3D]

The native proposal layer works really well, thank you for spending the time implementing that.
But I notice it is CPU only and it is taking about 2 seconds to evaluate an image using my native C++ code. My question is:

1. Are there plans to do a GPU implementation of the native proposal layer in the next release?
   or
2. is there some other way to get better evaluation performance, e.g. by mixing GPU and CPU layers for example?

[cha-zhang]

We got no time for this. For this iteration, what we hope to deliver is full disparity with Caffe for Faster RCNN. Please see our iteration plan here:
#2410

[Dozer3D]

Cha-zhang , thank you for the reply, at least I know.

If anyone knows how to get GPU performance for faster-rcnn inference on a native c++ windows application (without shipping python with our product), please speak up :-) !!.
I am assuming this won't be possible without the GPU proposal layer implementation?

[ahoink]

I had the same error when trying to load the fasterRCNN model in C++. I solved the issue by modifying FasterRCNN_config.py and changing the following line:
__C.STORE_EVAL_MODEL_WITH_NATIVE_UDF = False
to
__C.STORE_EVAL_MODEL_WITH_NATIVE_UDF = True
which produces a model with native UDFs