How can i obtain the attentional weights of feature interaction in AFM?

[chenfu12138]

No description provided.

[shenweichen]

Hi @chenfu12138 there are 2 steps to get the attentional weights of feature interactions in AFM.
First,make sure you have installed the latest release version of deepctr(now v0.1.5) from pip.
You can installed that version through pip install deepctr==0.1.5.

Step1: Modify two lines of the source codes

Please modify the layers.py in your local machine,maybe the path is
xxx\Anaconda3\Lib\site-packages\deepctr\layers.py

In line 134 and 135
change the following two lines

attention_weight =tf.nn.softmax(tf.tensordot(attention_temp,self.projection_h,axes=(-1,0)),dim=1) 
attention_output = tf.reduce_sum(attention_weight*bi_interaction,axis=1) 

to

self.normalized_att_score=tf.nn.softmax(tf.tensordot(attention_temp,self.projection_h,axes=(-1,0)),dim=1) 
attention_output = tf.reduce_sum(self.normalized_att_score*bi_interaction,axis=1)

I will modify it in the next released version.

Step2: Get the attentional weights !

After you have finished training the AFM model.

from tensorflow.python.keras.models import Model
from tensorflow.python.keras.layers import Lambda

afmlayer = model.layers[-3]
afm_weight_model = Model(model.input,outputs=Lambda(lambda x:afmlayer.normalized_att_score)(model.input))
attentional_weights = afm_weight_model.predict(model_input,batch_size=4096)

You can try it~

[chenfu12138]

@shenweichen It worked,thanks! I have obtained the attentional weights like this
[[0.05291238]
[0.05570798]
[0.29247418]
...
[0.2924742 ]
[0.05001876]
[0.04996691]]
But i dont konw how to match the attentional weight in the array with the specific feature interaction one by one,And it seems that the match relationship between them varies from every training?

[shenweichen]

Hi @chenfu12138
You can use the following codes

import itertools
import deepctr
from tensorflow.python.keras.models import Model
from tensorflow.python.keras.layers import Lambda

feature_dim_dict = {"sparse": sparse_feature_dict, "dense": dense_feature_list}
model = deepctr.models.AFM(feature_dim_dict)
model.fit(model_input,target)#

afmlayer = model.layers[-3]
afm_weight_model = Model(model.input,outputs=Lambda(lambda x:afmlayer.normalized_att_score)(model.input))
attentional_weights = afm_weight_model.predict(model_input,batch_size=4096)
feature_interactions = list(itertools.combinations(list(feature_dim_dict['sparse'].keys()) + feature_dim_dict['dense'] ,2))

The attentional_weights[:,i,0] is the feature_interactions[i]'s attentional weight of all samples

Try it and star it if it helps 😉

[chenfu12138]

@shenweichen Perfect solution! Its so nice of you! Star it of course!!!

[shenweichen]

Hi, the latest version v0.2.0 has been released ,please upgrade through pip install -U deepctr ~