'CLIP' object has no attribute 'clip_project'

[AhmedEwis]

I am trying to implement a custom clip:

model = clip_model.eval() 
device = CUDA(0) if is_gpu else "cpu"
model = clip_model.to(device)

use_beam_search = False 

import pandas as pd
prefix_length=10


df = pd.DataFrame([],columns=['image','desc'])
for fname in [i for i in os.listdir('image') if i.endswith('.png')]:
  image = io.imread('image/'+fname)
  pil_image = PIL.Image.fromarray(image)  
  image = preprocess(pil_image).unsqueeze(0).to(device)

  with torch.no_grad():
      prefix = model.encode_image(image).to(device, dtype=torch.float32)
      prefix_embed = model.clip_project(prefix).reshape(1, prefix_length, -1)
      
  if use_beam_search:
      generated_text_prefix = generate_beam(model, tokenizer, embed=prefix_embed)[0]
  else:
      generated_text_prefix = generate2(model, tokenizer, embed=prefix_embed)

  df.loc[len(df)] = [fname,generated_text_prefix]
  print(generated_text_prefix)

Getting error in the below code:

#prefix_embed = model.clip_project(prefix).reshape(1, prefix_length, -1)

ModuleAttributeError: 'CLIP' object has no attribute 'clip_project'

[vinson2233]

I think you confuse the CLIP model implementation from OpenAI repo and this repo.
In this repo, the clip model is defined using this class

CLIP_prefix_caption/train.py

Line 220 in 1ad805a

class ClipCaptionModel(nn.Module):

That's why if you load the CLIP model using OpenAI's implementation, they don't have the clip_project attribute