Help Me Bind SD.cpp to Godot on Your Platform With Your link.txt file!

[ganelonhb]

Hello friends,

I'm working on a project that binds Stable Diffusion to Godot. I need help with libs to link for various platforms that I don't use. If you have one of the platforms listed, it would be a huge help if you could please share a file in your build here in this discussion.

After building with the backend, the file I need should be located in stable-diffusion.cpp/build/examples/cli/CMakeFiles/sd.dir/link.txt

I need the following combinations of platform/backend:

LinuxBSD

• CPU
• CuBLAS
• HipBLAS

Windows (Visual Studio, it's just the better option).

The file I need is located in stable-diffusion.cpp\build\examples\cli\sd.dir\Release\sd.tlog\link.command.1.tlog

• CPU
• CuBLAS
• HipBLAS

MacOS

• CPU
• Metal

Please note that the copy of stable diffusion that is in my repo is fairly heavily modified and based on PR #386, but you should be able to just use the main branch of this repo unless Green-Sky replies here that his PR links new libraries.

Your help is greatly appreciated, and will advance the cause of bringing Stable Diffusion to video games. Thanks!

[Green-Sky]

In the long term ggml did some internal backend changes, that allow for loading backends at runtime, in an abstract way. We should update sd.cpp to latest ggml and port the code over to ggml-org/llama.cpp#9707 , that way you would "only" need backend specific ggml .ddls eg. .

Waiting on @leejet though to merge a couple prs for now...

[Green-Sky]

On a fun side note, @bssrdf ggml-org/ggml#971 has been working on making convolution faster and/or lighter on vram. I think there are still some open licensing issues though, but I guess you could include the license from the project the code was inherited from as a stopgap.

Also I did an experimental merge of my flash attention, the server and the winograd patches to test some stuff: Green-Sky#2

[ganelonhb]

That's great news. I'm especially excited for the winograd algorithm, since currently the only way I can see Stable Diffusion being actually used in videogames are if all the generating occurs during a loading screen... The more of the model I can fit in VRAM at a time the better. I am also experimenting with detecting multi-gpu setups and trying to force GGML to use one that is not in use by Godot, I think it's very possible but in my time learning the stable diffusion codebase, I haven't spent much time in GGML.

Still, for the time being I would like those build files. You wouldn't happen to have an AMD GPU or a M1 mac lying around to help me get those build files, would you?

[Green-Sky]

No I don't have anything that you are missing.
But I guess what you really want is proper CI/CD setup for all platforms 😅

[wandbrandon]

I have an M3 Mac and would love to help, need support for sd in godot for a research project! Besides this repo, any other way to get sd in a game engine? Here is the Metal and CPU link.txt

link.txt

[ganelonhb]

As far as I know, there is only an extension that connects to a horde (someone else's GPU). It can't really be trusted to always work correctly. I have been busy with other work, but as soon as I can get back to working on stable-diffusion.gd I will add this to the platform. In the meantime, if you need to get it up and running on your Mac, you can probably figure it out by looking at the SCsub file in my repo. At the very end of the file is the unconnected macos code. You want to just look at the windows and linux code and compare it to your link.txt file. What gets linked? Add it to the env variable. Scons uses python, so you can do anything that's legal in Python in Scons. Good luck! Your contribution is very much appreciated, I was really hoping a mac user would give me that file!