Text feature extraction

[Yangaiei]

Hi, Congrats on the amazing work. How to use CLIP to extract QVHIGHLIGHTS text features ? Can you provide the specific code?

[jayleicn]

Hi @angaiei007, please follow https://github.com/openai/CLIP#usage to extract the text features, this specific line below deals with text feature extraction:

text_features = model.encode_text(text)

[Yangaiei]

Hello,
1、I found that you have modified the CLIP.

2、I used your clip file. The data dimensions are the sam, but the value in the data has a little error.
Such as: {"qid": 8056, "query": "Chef makes pizza and cuts it up."}

code:

print("========  my   ===")
text_features = np.load('after_feature.npz')
print(text_features.files)
print(text_features)
print(text_features['last_hidden_state'].shape,text_features['pooler_output'].shape)
print(text_features['last_hidden_state'])

print("========you   qid8056.npz   ===")
text_features = np.load('u_clip/qid8056.npz')
print(text_features.files)
print(text_features)
print(text_features['last_hidden_state'].shape,text_features['pooler_output'].shape)
print(text_features['last_hidden_state'])

result：