-
-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature Request: IPAdapter and Latent Upscale (Neural Network) #37
Comments
Adding some form of image (re)mixing, iterations and upscaling would be a HUGE boon, one of my favorite features of MJ is the image remixing and iterations, give it an image you like and ask it to make similar ones or mix someone with a blobfish for hilarious results. Upscaling is just the icing on the cake. Would be very interested to see if this can be implemented. |
@Ptbird1 Simplest way to create variations of existing image(s) are img2img and Revision modes. If you want output images to look more like original you can just lower denoise in Image-2-Image settings. In latest version you can also use Canny Edge Control-LoRA (works as ControlNet - see CN tab), which is also great way to produce variants of original image :). |
@MoonRide303 Thanks for the prompt (hehe) reply! I have been AFK for a week so didn't know you implemented canny Cn!! That's so cool and I will definitely be using that once I'm back at my PC! I've used img2img a bunch and love it but what I was referring to was mixing two pre-existing images into a new generated image that contains the features of both originals. See photo |
@aimarxjg It should automatically pull update from github when you start it via |
It is feasible; actually, it is already happening; people are already using it for SDXL on comfyui (only works there). All the links I've posted are for SDXL (check ip_adapter_sdxl_demo.ipynb on the tencent link). Ipadapter for SDXL is one of the hottest things happening in the scene right now. And there is a need for an upscaler on Foocus/MRE, or at least something similar to the "highresfix" (from Auto1111), to correct eyes, etc., and then upscale.
Yep, that is exactly it. Your baby yoda-cat example illustrates it nicely. Here is another example: |
@vcm07 I like those results, but problem is I don't see IPAdapter in base ComfyUI code - and I'd rather not pull any custom nodes not maintained by comfy. I can get back to it when IPAdapter will be available in comfy backend - I've added feature request for that on ComfyUI side (ComfyUI issue #1392). |
@aimarxjg If MRE will find those files in proper folders then he won't try to download them, again. So if you already have those files, or can download it in a different way faster - feel free to speed up the process, and just put them in proper places :). |
No problem. I agree. Watching more closely, I see that most of these workflows and custom nodes with Ipadapter are very volatile. It happened a few days ago when Comfy updated recently; it broke everything, even the workflows, and these people (third parties) do add lots of new stuff (new custom nodes), change a lot of stuff, and this could create nightmarish upgrade problems, so you are right, it would be better to wait for an official implementation. Thanks for adding the feature request on the ComfyUI side. |
Upscaling is now possible in v2.0.12 via Image-2-Image - see example in Wiki. |
So, we are almost there, Fooocus with IPAdapter: lllyasviel#557 MRE with this = perfection; |
@vcm07 Those new features are still being stabilised in vanilla, but I will look into merging it within next couple of days. |
Fantastic! Thanks for all the work you're doing |
Ipadapter is a better and more robust Revision with more control. It can work as a real-time 1 image LoRa. People are doing revision + ipadapter and other combinations using Comfy. It would be really nice to have this in MRE.
ComfyUI Custom Node: https://github.com/laksjdjf/IPAdapter-ComfyUI
Link to the model: https://huggingface.co/h94/IP-Adapter/tree/main/sdxl_models
About IPAdapter: https://github.com/tencent-ailab/IP-Adapter
===
Upscale
SAG on Fooocus helps a lot, but MRE would need an effective upscale strategy to be even better. Some ideas:
The attempt by city96: https://github.com/city96/SD-Latent-Upscaler
By WAS/Ttl: https://github.com/Ttl/ComfyUi_NNLatentUpscale
These are some suggestions to create an upscaler for MRE; it would make this project even better.
Thank you, MoonRide, for this excellent project. You made Foocus much better, and thank you for the Colab edition. It's good for us folks that don't have a proper computer to run SDXL, and I'm using it every day.
The text was updated successfully, but these errors were encountered: