prototype progress api

[evshiron]

No description provided.

[Keavon]

Thanks for adding this! Could you please also make it return the latest image preview showing the in-progress generation? (It's what you get from turning up this range slider in the settings tab:)

[evshiron]

@Keavon This should work now.

Call /sdapi/v1/txt2img like this:

{
  // other parameters
  override_settings: {
    show_progress_every_n_steps: 5,
  },
}

And /sdapi/v1/progress will respond with a valid current_image field.

[Keavon]

That's wonderful, thank you @evshiron! I will have to give this a try shortly.

(I assume it also works for the img2img endpoint and your /sdapi/v1/txt2img was just an example, right?) I look forward to porting this over to the new API now in my image editor Graphite, which previously I had locked on this repo's commit 7d6042b using the unstable Gradio "API", or lack thereof.

[evshiron]

I assume it also works for the img2img endpoint and your /sdapi/v1/txt2img was just an example, right?

Yes, it should as they share the same code, but I didn't try yet.

Btw, Graphite looks amazing.

[Keavon]

Thanks @evshiron! Also real quick, before I port over the old Gradio "API" to this, I want to confirm all the features are supported. Since I have been pinned to that old version, I haven't tried out the API at all yet while I waited on this feature to be added (so thank you again for adding it!). If you don't mind, could you let me know if it currently provides access to all of these settings?:

• Seed
• Width and height
• Sampling steps
• Sampling method
• img2img denoising strength
• CFG scale
• Negative prompt
• Fix faces
• Tiling

Those are the settings we expose (some of them renamed for better user-facing intuitiveness.)

[Keavon]

I also just had a thought: in your API endpoint, it might be useful to add an optional parameter in the GET request that can disable
(or enable?) the base64 image in case you want to poll for more frequent percentage progress updates but only download the base64 image less frequently.

[evshiron]

To be honest, I haven't tried all of the listed features, but the parameters in txt2img and img2img APIs are actually constructing this class, which should cover the options in txt2img and img2img tabs of WebUI.

And after #3629, even options in Settings tab can be overrided for a run.

So I would say these APIs should satisfy your need.

I also just had a thought: in your API endpoint, it might be useful to add an optional parameter in the GET request that can disable
(or enable?) the base64 image in case you want to poll for more frequent percentage progress updates but only download the base64 image less frequently.

Yes, there was one, I will add it back.

[Keavon]

Perfect, thanks! I look forward to hooking it up as soon as I get a few hours free.

Also, can this progress endpoint be used to check for server connectivity, without potentially interfering with the progress of another user that's currently generating an image? I have a feature that checks that it can reach the server when you type in an IP address/hostname, before it generates anything.

I assume there isn't an actual "server status" endpoint yet which is why I am wondering if the status endpoint would be sufficient. Or if there might be any other sort of dummy endpoint, even if it returns a 400-level code at least I'd know the server exists. Ideally I'd like to also use that to query for server information, like available VRAM (so I can automatically pick a maximum resolution allowed for image generation).

[evshiron]

Also, can this progress endpoint be used to check for server connectivity, without potentially interfering with the progress of another user that's currently generating an image?

I am new to this project, but there is a queue_lock, so I assume at normal circumstances only one task is allowed at the same time. I might be wrong though.

Or if there might be any other sort of dummy endpoint, even if it returns a 400-level code at least I'd know the server exists.

I send a HEAD request to /sdapi/v1/txt2img and expect a HTTP 405.

I am doing an alternative UI too, but I don't know if a server status API will be needed, or maybe someone else to implement it.

[Keavon]

Looks good! I was thinking about the pros and cons of defaulting to showing the image automatically (with a skip option) or not showing it automatically (and requiring the opt-in send option). I came to the conclusion that it's probably better API design to maintain consistency and require it be opt-in:

• If show_progress_every_n_steps is off, it will not respond with the base64
• But if it's accidentally on, all requests suddenly get unexpectedly slow due to networking speeds
• However you can opt in to skip it, bringing us back to the situation we had when show_progress_every_n_steps was off

That's basically three levels of off by default (except by request) -> on by default (except by request) -> off (by request). It's probably more consistent and safer to have the image off by default, so it's off by default (except by request) -> off by default (except by request) -> on (by request).

That would mean renaming skip_current_image to send_current_image and inverting it, so it defaults to off if not provided. That way API requests don't end up being heavily inundated by unintended base64 image data unless it's explicitly requested. It would be ideal to also send a descriptive error in the event that send_current_image is provided but show_progress_every_n_steps is off.

Do you have thoughts about that?

[evshiron]

My thought is that if we are calling APIs to generate images, we might already explicitly override some settings (for example if we want image preview, we set show_progress_every_n_steps), so a valid current_image field might be expected. But if suddenly we don't want that field any more, we can "skip" it. I treat non-default parameters as modifiers though.

There might be changes when @AUTOMATIC1111 wants to merge this PR.

[bamarillo]

I was thinking about this today, but I don't think spamming the backend with requests for an update is the correct thing. I understand that FastAPI comes with websocket support out of the box. What do you think about the client subscribing to a websocket and then the backend can push updates to the client in real time? The client also can control the rate of the updates by just ignoring the messages it doesn't want.

[Keavon]

I was thinking about this today, but I don't think spamming the backend with requests for an update is the correct thing. I understand that FastAPI comes with websocket support out of the box. What do you think about the client subscribing to a websocket and then the backend can push updates to the client in real time? The client also can control the rate of the updates by just ignoring the messages it doesn't want.

This would probably be useful, but I'd suggest keeping both as options so different use cases can benefit from the simplicity of polling or the efficiency of websockets. But I'd suggest leaving that for a separate PR so it doesn't block this getting merged.

[Keavon]

Thank you @AUTOMATIC1111!

[evshiron]

Thank you @AUTOMATIC1111 for your rework in 1497842. But state.time_start is missing now (which is taken from here), and the progress API is broken at the moment.

I wonder we can add that back to state.begin(), or do you have any thoughts about that?

EDIT: #3970

[evshiron]

I was thinking about this today, but I don't think spamming the backend with requests for an update is the correct thing. I understand that FastAPI comes with websocket support out of the box. What do you think about the client subscribing to a websocket and then the backend can push updates to the client in real time? The client also can control the rate of the updates by just ignoring the messages it doesn't want.

Yeah, you are right. Implementing this via WebSocket should be more performant and state of the art, but my ideas are:

1. SDWebUI might not be a project focused on API performance that much, and as there is only one task running at a time, polling via a HTTP endpoint should achieve OK performance.
2. I am not a Python expert and I am imitating to contribute most of the time. It's a bit hard for me to introduce a WebSocket implementation and decide the data schema to be used for now or maybe the future. So I decide to do a "prototype" PR.
3. I agree with @Keavon that if someone needs WebSocket, let's do that in a separate PR.

[Keavon]

@evshiron I'm working on integrating it with Graphite now by updating the old Gradio "API" endpoints. I realized I'm missing an endpoint to terminate the job, so I'm sticking with the Gradio endpoint on that for now (but if you might be willing, that would be superb if you could do another PR to add that endpoint, or anyone else who might be reading this). I hooked it all up but I'm seeing a response of null for the response image. Do you have any idea why? Details below.

This is running the commit 060ee5d3a7ba258f944d0c891f90ac4a65411446 of your merged PR, before the rework commit that came in after.

Initial request:

Progress request:

Thanks for the assistance!

[evshiron]

@Keavon I roll back to that commit and it works on my end.

I use a setInterval to continuously call the progress API. The current_image should be set as it progresses.

[evshiron]

I try your parameters and they work too.

Try adding some logging around here?

stable-diffusion-webui/modules/api/api.py

Line 190 in 060ee5d

current_image = None

See also here:

stable-diffusion-webui/modules/sd_samplers.py

Line 106 in 060ee5d

if not shared.parallel_processing_allowed:

[Keavon]

current_image = None
print("shared.state.current_image:", shared.state.current_image, "req.skip_current_image:", req.skip_current_image)
if shared.state.current_image and not req.skip_current_image:
    current_image = encode_pil_to_base64(shared.state.current_image)

It spams this when generating:

shared.state.current_image: None req.skip_current_image: False

---

if opts.show_progress_every_n_steps > 0 and shared.state.sampling_step % opts.show_progress_every_n_steps == 0:
    print("shared.parallel_processing_allowed:", shared.parallel_processing_allowed)
    if not shared.parallel_processing_allowed:
        shared.state.current_image = sample_to_image(decoded)

Spams this:

shared.parallel_processing_allowed: True

[evshiron]

stable-diffusion-webui/modules/shared.py

Line 107 in 060ee5d

parallel_processing_allowed = not cmd_opts.lowvram and not cmd_opts.medvram

Oh well, my graphics card isn't that good so I have --medvram enabled, which makes parallel_processing_allowed to be False.

WebUI has support for parallel_processing_allowed here, while API haven't:

stable-diffusion-webui/modules/ui.py

Line 280 in 060ee5d

if shared.parallel_processing_allowed:

I am going to see how that support can be integrated in the API part.

[Keavon]

Ah, that makes sense! Too bad that's currently a limitation. I have just enough VRAM (8 GB) for it to be viable to turn that flag off since it ~2-4x's the computation time, I found. But for larger images (approaching 1 megapixel) it breaks. Thank you for looking into that, I really appreciate it! Happy to help test with my hardware wherever I can be of assistance.

[evshiron]

Made a new PR #3970.

[Keavon]

I pulled your PR and it looks like everything is working now. Thank you for that! I guess now I just need the terminate endpoint and I'll have feature parity. I wonder if you plan to add a terminate feature to your UI so that'd be in line with your efforts? I could also file an issue and hope someone else takes it on— or I can poke into it myself once I'm not so busy after Halloween, although I don't know this codebase or Python at all. Anyways, thank you again for this :)

[evshiron]

@Keavon I made a prototype interrupt API here:

https://github.com/evshiron/stable-diffusion-webui/tree/feat/interrupt-api

If you are in a hurry you may want to try that out. Naive implementation though.

[Keavon]

@Keavon I made a prototype interrupt API here:

https://github.com/evshiron/stable-diffusion-webui/tree/feat/interrupt-api

If you are in a hurry you may want to try that out. Naive implementation though.

Splendid, that works well so far as I can tell. And I appreciate the simplicity of the mere four lines of code. Is there anything else you'd need to change about it before opening a PR?

[Keavon]

I send a HEAD request to /sdapi/v1/txt2img and expect a HTTP 405.

I'm using this too now, thanks. But the downside is that it adds...

...to the console and there's apparently no way to suppress that, despite being called from within a try-catch block. Do you think it would be appropriate to add an "always return 200" status check endpoint, @evshiron and @AUTOMATIC1111? Perhaps right now it could return an empty body but it could return other status information in the future (like available VRAM would be handy).

[evshiron]

How about GET /sdapi/v1/progress as it's landed now, which should always return HTTP 200 unless exception occurs.

I currently don't have a motivation for system status API though, and choosing which status to return seems to be a pain.

[evshiron]

Splendid, that works well so far as I can tell. And I appreciate the simplicity of the mere four lines of code. Is there anything else you'd need to change about it before opening a PR?

I don't know, it feels too simple to be a PR.

Added in #4025.

[Keavon]

How about GET /sdapi/v1/progress as it's landed now, which should always return HTTP 200 unless exception occurs.

That's a good idea, and seems to work so far. Thanks for the suggestion.

Thanks also for opening that PR, hopefully both can be merged soon!