I can't reproduce the results for Inference time mitigation

[liuxiao-guan]

I have tried to reproduce the results for Inference time mitigation. But I found that compared to the experimental results in the paper, the reproduced FID values are high and the similarity scores are low. Where could the problem lie?
To reproduce the results, I performed the following operations.

1. I first generated the pictures using the diff_inference.py
   python diff_inference.py -nb 4000 --dataset laion --capstyle instancelevel_blip --rand_augs rand_numb_add
2. Then, I evaluated the pictures
   python diff_retrieval.py --arch resnet50_disc --similarity_metric dotproduct \ --pt_style sscd --dist-url 'tcp://localhost:10001' --world-size 1 --rank 0 \ --query_dir /root/autodl-tmp/logs/Projects/DCR/inferences/defaultsd/laion/instancelevel_blip_auginfer_rand_numb_add_2/ --val_dir /root/autodl-tmp/laion_10k/train/

For the two python file, I only changed the path of the file，including savepath，checkpath(Stable diffusion 2.1) and prompt_json. And the dataset I used is laion10k which was given in your readme.
But the results were unexpected.


Fid=21.833.
The experiment in the paper is as shown below：


As you can see, the sim95_pc is 0.27, which is much different from the 0.556 in the experiment. The same is true for FID values.
Although this experiment was only run once, I feel that the results of one time should not be so different from the experiment.
Can you give me some advice on what the problem might be? Thanks.

[somepago]

The mitigation numbers are reported for a fine-tuned model (with ddf=5) on top of the SD-2 and laion-10k dataset.

Did you train that model and are inferencing on it?

[liuxiao-guan]

Oh，yeah, that is the problem, I just used the SD-2 to reproduce the result. Thanks! By the way, can the code also be run using accelerate? I saw that accelerate was used in the code

[somepago]

Thanks. Accelerate should work. Lmk if it doesn't.