"Real-time" ControlNet Scribble interface front-end.
Allows drawing in a browser and seeing results as they are generated by an active back-end (not included in this repo, see below).
Demo.webm.mov
This is a front-end toy that assumes Stable diffusion + ControlNet is exposed as an API with the schema used in AUTOMATIC1111/stable-diffusion-webui.
Requests are sent to http://127.0.0.1:7860/sdapi/v1/txt2img as is hardcoded in main.js, so using another endpoint requires editing that.
Because the landscape for Stable Diffusion + ControlNet changes quite rapidly, it may be difficult to keep an updated guide for the best possible back-end setup. A semi frozen fork of some valid implementation could be an option.
In the meantime, please see the links under Resources, or follow these informal steps:
- Install and run vladmandic/automatic (recommended because it's relatively smooth to set up, having some extensions incl. ControlNet out of the box). Alternatively, install AUTOMATIC1111/stable-diffusion-webui and Mikubill/sd-webui-controlnet.
- Test ControlNet in the installed webui, make note of the name of the "scribble" model you have, (e.g.
control_scribble-fp16 [c508311e]). If this is working properly, you can close it and continue with the next step. Otherwise review step 1. - Restart webui.bat or equivalent with command line args to enable api and allow a CORS origin for the front-end, e.g.
--cors-allow-origins=127.0.0.1:3000. If using vladmandic/automatic, api is enabled by default and the argument for CORS origins is--cors-origins, or runwebui.bat --helpand check. - Host index.html with the allowed ip, e.g. run
python3 -m http.server 3000in the directory of this repo. - Open http://127.0.0.1:3000 with your browser and make sure the correct model is supplied (edit the default in index.html to avoid repeating this), if the model is not found, your scribble will be silently ignored.