Example of using the transformer layer

[kevinlin311tw]

Hi Happynear,

I noticed that you have implemented the transformer layer recently. My understanding is that this layer is related to the spatial transformer layer from the recent arxiv "Spatial Transformer Network". It will be perfect if you could provide some examples of using this transformer layer.

Thank you
Kevin

[happynear]

Hi @kevinlin311tw ,

It is only an experimental semi-finished realization. It only supports affine transform and suffers from a very low speed. I exploited it before I went out for business, and I don't have time to continue work on it.
The codes is very easy to use. It takes a feature map blob and an affine tansform parameter blob, ouput the interpolated feature map. The affine tansform parameter can be got by a two-layer NN regression model as the paper describes.

It will be appreciated if you could take over the work, continueing to accomplish it.

[ducha-aiki]

@happynear, nice work!
I will try to develop it further. For the begging - bug report #35 :)

[wazhenzhen]

Hi, @happynear ,
I have a question about the regression layer (localization network). I implement it with two fully-connected layers, and the outputs of the second fully-connected layer is 6. But in their paper, they claimed that a final regression layer should be appended after NN. I'm puzzled about which kind of regression layer is (like L2)? And as we cannot access to the groundtruth of transformation \theta, how can we do regression?
Thanks a lot!

[happynear]

@wazhenzhen ,
The gradient of the second fc layer is from transformer_layer. The foumula of calculating the gradient is written in the paper, and that is the most important highlight of the paper. I have implemented the codes in transformer_layer.cpp and transformer_layer.cu.

[wazhenzhen]

@happynear , Thanks! As I'm a freshman on caffe, I'm still not clear about how to implement the whole spatial transformer network to a standard CNN. Could you share a .prototxt file to show how to use the transformer layer, it will be much clear I think.
Thanks a lot!

[happynear]

I am busy for cvpr deadline now. After that, I will continue to work on the transformer layer.