Issue:
The activation map and pooled vector(before fc layer) in Resnet-50 are non-negative. Then, no component is negative for dot product.
Potient solution:
1. Remove relu layer before pooling layer.
2. Based on equation in Grad-CAM:
w is weighs of fc layer. Yc is score for category c. A is activation map, k is channel, i_j is spatial location.
The equation is the goal of Grad-CAM, to find weighs for each activation map and combine them.
In similarity visializtion for embedding, the goal is the same, we try to find ∂D/∂A, which is the contribution for each activation map to the distance between two image vector. D is dot product of fc_vect_a and fc_vect_b. Since ∂D/∂A = ∂D/∂Y * ∂Y/∂A, (Y is fc layer vector) , and based on equation in Grad-CAM, we can get ∂D/∂A = fc_vect*W, which fc_vect and A from different images
So, we can use vector after fc layer and W matrix to get a new vector(which have negative value) to replace orignal pooled activation map.
3. we can forward vector in each location in activation map to fc layer. And use the output.
Passive Vision 