Expected that server's /v1/images/edits endpoint always forces img2img?

[wwoodsTM]

Hi! I was looking through and comparing what the server endpoint /v1/images/edits does to what CLI does, as well as what the official ComfyUI workflows do (at least for Klein and QIE), and I wanted to ask whether I have spotted what may be an important distinction?

So basically from what I can tell, current stable-diffusion.cpp server code forces init_image to be assigned and always copies it from the first reference image, which I believe in turn forces all /v1/images/edits requests to route through img2img (to be specific: this is what I saw in examples/server/routes_openai.cpp where it looks like decoded edit images are pushed into request.gen_params.ref_images and that first one is copied into request.gen_params.init_image).

My understanding from looking at the CLI side is that this behavior is not shared over there and looking through commit history it also looks like this was not the original behavior of the /v1/images/edits endpoint and came along with some of the refactoring that's been going on with the server code.

Now as a reference point for comparison with the official Comfy workflows it looks like this server behavior more or less matches up for QIE, but where things don't match I believe is for their Klein workflows. In this one, for example, it looks like the sampler’s latent_image is coming from EmptyFlux2LatentImage, which I assume would match up more with the old behavior for the sd.cpp endpoint and what is possible through the CLI if you don't explicitly pass an init image?

Now, I don't know this code base or comfy's well enough to confidently say much more than that, and I could totally be off here or maybe this isn't an important difference, so that's why I am posting this in "Discussions", but I at least wanted to throw this out in case I actually was on to something.

Anyways, thanks for all the hard work on this awesome project, I do appreciate it a ton!

[wbruna]

https://github.com/leejet/stable-diffusion.cpp/blob/master-767-885f01a/examples/server/routes_openai.cpp#L177

I agree, it looks like a bug.