query about feature maps

[ustundag]

Hi all, thanks for your great effort in this research! I have a question on how to get the feature map of an equirectangular image input.

For example, I used the model S2ConvNet_deep in the MNIST example code to get feature map of an equirectangular image by cutting out the linear block at the end. I resized input panoramic image to 100x100 and the initial bandwith is 50. But the model returns an output in shape (1, 64). How can I get larger feature map like a VGG feature extractor?

Can you please share your precious comments?

[mariogeiger]

Take out also the ˋso3_integrate` operation

[ustundag]

Hi @mariogeiger, thanks for your answer.

Without so3_integrate layer, it now returns a feature map in shape (1, 64, 12, 12, 12). How can I interpret last three (alpha=12, beta=12, gamma=12) dimensions as feature vectors? I mean, VGG16 returns 512 feature vectors from a feature map (1, 512, 7, 7).

[tscohen]

It's a 3D feature map, with axes for alpha, beta (spherical coordinates) and gamma (filter orientation).

[ustundag]

Thanks for your comment.

So, how can I convert the final 5D feature map(1, 64, 12, 12, 12) into a 4D feature map like VGG-16 (1, 512, 7, 7)?

I want to extract specific features from the feature map. For example, I have a panoramic image with the size of [10, 5]. An object located at the pixel [1,1] and I want the feature vector at [1,1] from the feature map.

I really appreciate if you share your precious comments. Thanks in advance. @mariogeiger @tscohen @jonas-koehler