Details about how to extract sd features

[GOZGOXGOYGO]

Hi Junyi,

I am confused about how to extract sd features. Actually the file extractor_sd.py seems to output a feature in shape of [1, 1280, 16, 16] without obvious semantic information. And it seems to use the model weights from project ODISE. Could you please provide a script to easily extract and visualize the sd features using publicly available stable diffusion model weights? Thanks a lot!

[Junyi42]

Hi, thanks for your interest in our work.

Please refer to this notebook for a detailed extraction and visualization.

We only use the weights of ODISE implicit captioner and its post-processor for segmentation, and the rest is SD-1-5 public weight. We are planning to provide the script only using the public stable diffusion weight, but unfortunately I'm in the middle of a few things and won't be able to update it before mid-December.

Best,
Junyi