Generate "OOOOOOOOOOOOOOOOOOOOO" instead of words

[SamuraiBUPT]

Hello, I follow the tutorial in instructblip and pick a test image. But its output seems to be weird.

Code:

from lavis.models import load_model_and_preprocess

test_img2 = Image.open('../dataset/fashion-new/images-raw/test_img.jpg').convert('RGB')      # open the image and convert to RGB
display(test_img2.resize((256, 378)))

model_instructBlip, vis_processors, _ = load_model_and_preprocess(name="blip2_vicuna_instruct", 
                                                                    model_type="vicuna7b", 
                                                                    is_eval=True, 
                                                                    device=device)

image_ = vis_processors["eval"](test_img2).unsqueeze(0).to(device)
caption_test = model_instructBlip.generate({"image": image_, "prompt": "Describe the man's clothes."})
print(caption_test)

output:

['OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO']

I wonder what step I missed or did wrong to get this result.

Any help will be appreciated.

[SamuraiBUPT]

Problem solved, I used vicuna-7b-v1.5, not vicuna-7b-v1.1, sorry to bother.

[purshow]

Hi bro!I meet the same question.Could you teach me how to solve it?

[SamuraiBUPT]

Hi bro!I meet the same question.Could you teach me how to solve it?

The reason my model generated "OOOOOOOO" is that I was using vicuna-7b-1.5, but actually, according to the document of LAVIS, you should use vicuna-7b-1.1 instead. The instruct-blip model seems not to be capable with vicuna version 1.5.

Please check your vicuna version to avoid using model of wrong version.

[purshow]

Thanks！