Why feed different input (same label) to the network get different grad_val?

[taylover-pei]

When I feed different image, which share the same label, to the network, I check the grad_val. But, I find that, the grad_val corresponding to different images are different. Actually, I think, for the same class, the grad_val should be the same, which is related to the parameters.

So, how should I solve the problem?

Looking forward to your reply.

[jacobgil]

The gradient computed is of the category score, with respect to the activations. The output from the network layers (the activations) is different for different images, so the gradients should be as well.

[taylover-pei]

Thank you for your reply.

Since the gradient is is derived from the differentiation of the category score against the activations. Although the activations is different for different images, the category score is different, either.

So, I think the gradient is only related to the parameters of the network. Am I right?

For example, when the GAP layer is followed by a single fc layer. The gradients for different activations belonging to different images are the same. However, when the GAP layer is followed by two fc layers. The gradients for different activations belonging to different images are different.

Thank you.