Problem when running demo.ipynb (blank result)

[Sutongtong233]

Hi, I am interested in this great job! Here I have a problem when running demo. I use LLaMA-1b-hf and vqgan-f16-8192-laion(As shown in data_preparition). The generated_img results:

(I find generated_img.max() = 0.07. Is there any mistake?
Thanks

[ggjy]

Hi, I reran the demo.ipynb and it works in my environment. I've uploaded the corresponding vqgan model here, so maybe you can try out this one?

[Sutongtong233]

Thanks! but I still face the problem. Here are some intermediate results:

• after vqgan encode

• after model.generate

[ggjy]

Here are some intermediate results from my environment:

What about your input_ids? It's strange that the final output sequences seem different between our results.

[Sutongtong233]

The input_ids are the same.

I print outputs.sequences[:, -256:], since the beginning of outputs.sequences are input_ids:

Are you using the LLaMA-1b of LLaMA-300M?

[ggjy]

I use the LLaMA-1b, and the transformers version is 4.38.1. Maybe there is something wrong when loading the ckpt?

[Sutongtong233]

When I change to other images, the results is correct. Strange :< Thank you very much~

[Fangyi-Chen]

Hi,

Thank you for your great work. I am facing the same problem . Using the seg_1.png gives black and my inputid / output.sequence is the same as [Sutongtong233]'s. When I screenshot the seg_1.png, it works reasonably. Other images under the data folder are good as well.

I am using pytorch 1.13.1 and transformer 4.38.1. Could you give any advice to debug this? Or any insight that could cause this strange issue? Thank you and looking forward to your reply.