Add ControlNet

[Warvito]

Implements #296

[marksgraham]

Nice Walter, looks good to me!

As I understand it, the ControlNet needs to be initalised with the same weights as the trained diffusion model. Should we provide a helper function for that? If you agree, we could do it in another PR as I know @virginiafdez is already working on a tutorial with this controlnet and don't want to hold up its merging anymore

[virginiafdez]

Hello,

For now, I''m not working on a GitHub branch of the repo, but offline as I need to make sure the model works and such. But yeah, I think maybe it's better to do it like @marksgraham says !

[Warvito]

Nice Walter, looks good to me!

As I understand it, the ControlNet needs to be initalised with the same weights as the trained diffusion model. Should we provide a helper function for that? If you agree, we could do it in another PR as I know @virginiafdez is already working on a tutorial with this controlnet and don't want to hold up its merging anymore

Thank you for the suggestion, @marksgraham . I agree, I think it will be great to have helpers to initialise the controlnets