Possible bug in ControlNet code #126
Replies: 4 comments 3 replies
-
Did you fix this? I also found the ControlNet result is not right compared to the paper, is it possible causing the problem? Did you test it ? |
Beta Was this translation helpful? Give feedback.
-
This issue may cause problem when the input is image, not HED. We have fixed the bug in the ControlNet in PixArt-Sigma. Will released soon. Stay tune! |
Beta Was this translation helpful? Give feedback.
-
I have found the same problem as you, that is, there is a slight difference between the process illustrated in the paper and the actual code. What I'm thinking is that since the copied block is trainable, the difference may not be significant? |
Beta Was this translation helpful? Give feedback.
-
I am currently trying to use Pixart alpha to train ControlNet on my own dataset, with a training resolution of 1024 and a task of image restoration (condition is image). If there is any progress, I will synchronize it here |
Beta Was this translation helpful? Give feedback.
-
Hey,
I have a question regarding the PixArt ControlNet code.
The paper suggests to add the input directly to the conditioning signal in the trainable network copy. However in the code, you first forward the input through the first frozen block and only then add it to the conditioning.
The first trainable copy therefore does not receive the input as expected, but the already modified and slightly encoded input from the first frozen layer. This seems unintuitive. Is it possible that this is wrong behaviour or did I overlook something?
Beta Was this translation helpful? Give feedback.
All reactions