Inconsistency of classifier-free guidance between training and testing.

[TianpengBu]

HI, authors,
Great work! My question about the implementation is as follows:

During training, I found that you randomly set 20% of CLIP's input as zeros tensors,

however, during testing, you concatenate the output of clip embedding with zero tensors, like this:

As far as I am concerned, to align the training and testing, should we randomly set 20% of the output of CLIP as zero tensors rather than the input of CLIP model?

[jiangzhengkai]

@TianpengBu I totally agree with you