Support for GlobalPooling layers?

[ConjugateDual]

Keras's implementation of Inceptionv3 (in keras.applications) uses a GlobalAveragePooling2D (or alternatively a GlobalMaxPooling2D) layer as its penultimate layer. Global pooling layers in keras extend _GlobalPooling1D, etc., instead of _Pooling1D, etc., so they fail the class membership test
if isinstance(layer, (_Conv, _Pooling1D, _Pooling2D, _Pooling3D)) in vis.visualization.visualize_cam.

Really, though, that's just a subproblem to my real issue. My bigger issue is I can't figure out how to modify the method to create a meaningful heatmap. The problem is that the GlobalPooling layers in keras have are a 2D tensor of shape (batch_size, channels) instead of a 3D tensor like the other Pooling layers used in VGG16, for instance (see e.g. https://keras.io/layers/pooling/). Hence, the
output_dims returned by utils.get_img_shape(penultimate_output)[:2] in visualize_cam is an empty tuple (), so the heatmap is a scalar. This of course creates problems when trying to resize the heatmap to the input shape, which will be an array of (pixel_height, pixel_width), depending on the input shape of the model.

Do you have any guidance for this issue?

[mhubrich]

Instead of using the GlobalAveragePooling2D layer as the penultimate layer, you could use the layer just before (usually a Concatenate layer in case of Inceptionv3).

I'm using the Inception models as well and I encountered no problems by doing this.

[raghakot]

Ideally, you should not use a layer that has a spatial resolution of (1, 1) or (1). This will just give a constant value on resize and is not very useful in capturing spatial attention. To get good grad-cam visualization try to use layer which has resolution > (1, 1). @DepthFirstSearch suggestion should work.

I think it is correct to exclude _GlobalPooling in membership check. Let me know if you feel otherwise. Feel free to reopen if you want to add anything else.